Commit Graph

55 Commits

Author SHA1 Message Date
Nathan.fooo 30f87457f3
feat: support uploading file with multiple part (#625)
* refactor: replace crate

* chore: remove s3-rust crate

* chore: test multiple upload

* chore: client api

* chore: client api

* chore: v1 version upload,download

* chore: fix test

* chore: fix test

* chore: remove workspace test

* chore: fix wasm build

* chore: update path

* chore: update path

* chore: rename

* chore: expose file entity

* chore: introduce client-api-entity

* chore: fix build

* chore: new url

* chore: fix test

* chore: merge main

* chore: fix clippy
2024-06-19 13:06:58 +08:00
Bartosz Sypytkowski d9354202cd chore: specify chunk size for search and index 2024-06-18 11:45:33 +02:00
Bartosz Sypytkowski ae2a2a4fa0 chore: appflowy indexer uses appflowy ai as open ai proxy 2024-06-18 10:04:38 +02:00
Bartosz Sypytkowski f11703949c chore: turn off sqlx macros in tests 2024-06-18 09:34:10 +02:00
Bartosz Sypytkowski a5bca1852b chore: change test query to macro 2024-06-18 08:53:02 +02:00
Bartosz Sypytkowski 9ca904f943 chore: change open ai token tracking to daily basis 2024-06-18 06:53:39 +02:00
nathan f0a32330b2 chore: fix clippy 2024-06-17 22:08:02 +08:00
Bartosz Sypytkowski 3c72f1292d chore: fixed tests for document content extraction 2024-06-14 13:01:16 +02:00
Bartosz Sypytkowski 7ad105ee34 fix: index all of the document blocks 2024-06-14 11:23:19 +02:00
Bartosz Sypytkowski b52369de7f
feat: add workspace settings with disable indexing option (#606)
* feat: add workspace settings with disable indexing option

* chore: added api workspace settings endpoint

* chore: add workspace settings read and write to client api

* chore: add tests for workspace settings client api

* fix: enforce authorization rules on workspace settings api

* chore: fix appflowy indexer test configuration

* chore: add dotenvy to appflowy indexer tests

* chore: override appflowy indexer env vars for tests

* chore: add appflowy indexer preindex env var
2024-06-06 09:26:21 +08:00
Bartosz Sypytkowski 58e01f4e8d
fix: index documents with missing embeddings at service start (#597)
* feat: appflowy-indexer indexes documents with missing embeddings at start

* chore: add tests to check initial indexing of collabs

* chore: log user search requests and response

* fix: wrong appflowy openai api key env var name

* chore: fill openai keys for integration testing

* chore: silence flaky test
2024-06-04 16:50:06 +08:00
Nathan.fooo 1cc5b58254
chore: update client api (#593) 2024-06-02 20:20:14 +08:00
Nathan.fooo 9b4bfce616
chore: fix ai client test (#589) 2024-05-30 07:50:25 +08:00
Bartosz Sypytkowski 128e978016
chore: add appflowy indexer service start to cargo run xtask (#586) 2024-05-29 23:45:30 +08:00
Bartosz Sypytkowski c4702bbbdf
feat: Workspace token usage (#584)
* feat: register open ai token usage during indexing

* feat: register open ai token usage during search

* chore: fixed open ai token usage when searching for documents
2024-05-29 16:07:56 +08:00
Bartosz Sypytkowski d3e59fa801
feat: Appflowy-indexer service and /api/search endpoint (#582)
* feat: implement appflowy-indexer and search api endpoint

* chore: store embedding content type

* chore: more descriptive messages for open ai missing key

* chore: fix sqlx errors in tests

* chore: document watcher - change plain text rendering to use depth-first scan

* chore: added nested document blocks example
2024-05-28 21:09:29 +08:00
Nathan.fooo 3f2d5f0785
chore: optimize redis stream space usage (#579)
* chore: set maxlen for redis stream

* chore: update log

* chore: set expiration time
2024-05-27 13:22:14 +08:00
Khor Shu Heng 93b4a1516c
feat: add postgres listeners for collab service (#575) 2024-05-27 09:12:32 +08:00
Nathan.fooo 7284430cad
chore: add test (#571) 2024-05-23 21:03:36 +08:00
Nathan.fooo 4eb1e6bceb
chore: generate history using redis steam (#570)
* chore: combine test

* chore: add snapshot test

* chore: add test
2024-05-23 07:57:20 +08:00
Nathan.fooo 05e7f1cda7
Integrate ai chat (#569)
* chore: fix compile

* chore: implement client api

* chore: implement client api

* chore: clippy
2024-05-22 11:18:07 +08:00
Khor Shu Heng 67d9fad7d7
chore: migrate collab storage to appflowy-collaborate (#566)
* chore: migrate collab storage to appflowy-collaborate

* fix: clippy error

* chore: remove handler

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-21 10:06:26 +08:00
Nathan.fooo 2736fa60a7
chore: integrate redis stream (#559)
* chore: integrate redis stream

* chore: make CollabUpdateStreaming::send_update synchronous

---------

Co-authored-by: Bartosz Sypytkowski <b.sypytkowski@gmail.com>
2024-05-18 08:28:57 +08:00
Khor Shu Heng fbdcbd6e08
chore: migrate collab actors to appflowy-collaborate (#556) 2024-05-17 12:22:43 +08:00
Khor Shu Heng c79267d42a
feat: migrate collab access control to collaboration service (#548)
* feat: migrate collab access control to collaborate service

* feat: migrate workspace access control to separate crate
2024-05-15 15:24:07 +08:00
Khor Shu Heng d9d874ab8a
feat: migrate collab cache to collaborate service (#524)
Co-authored-by: nathan <nathan@appflowy.io>
2024-05-09 23:12:13 +08:00
Nathan.fooo 388d38428b
chore: revamp post realtime endpoint (#536)
* chore: revamp  post realtime endpoint

* chore: fix clippy

* chore: fix fmt
2024-05-08 22:06:37 +08:00
Nathan.fooo 70262a1ac4
chore: try to not block actor mailbox (#534)
* chore: add test

* chore: spawn task to handle message

* chore: update error message
2024-05-08 16:54:05 +08:00
Nathan.fooo 3508262d1d
chore: using env to control mailbox size (#532)
* chore: using env to control mailbox size

* chore: update logs
2024-05-07 22:15:53 +08:00
Bartosz Sypytkowski 47e6f1e8b1
Update Yrs and AppFlowy-Collab (#518)
* chore: updated collab dependencies to v0.2

* chore: update collab to v0.2 and yrs to v0.18

* chore: pinned collab to specific revision

* chore: fix clippy errors

* chore: update commit id

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-06 11:30:02 +08:00
Nathan.fooo eba2e0d4db
ci: add test (#516)
* ci: add test

* ci: add test

* ci: add readme

* ci: update test
2024-05-02 21:30:23 +08:00
nathan 60967e51e6 chore: disable gc 2024-05-02 07:52:38 +08:00
Nathan.fooo 59b3d69c42
chore: fix local set warnings (#515)
* chore: fix local set warnings

* chore: clippy
2024-05-01 23:07:57 +08:00
nathan ca1623fcc2 chore: update logs 2024-05-01 22:05:04 +08:00
nathan 885e6f31b4 chore: enable gc 2024-05-01 21:00:40 +08:00
Bartosz Sypytkowski 29928cb4a7
chore: add more detailed logging around errors in Transaction::try_encode_state_as_update_v1 (#514) 2024-05-01 20:14:27 +08:00
Nathan.fooo 79bb510aaf
ci: run history test (#513)
* ci: run history test

* chore: fix ci
2024-05-01 16:28:07 +08:00
Nathan.fooo e2fd049333
chore: awarenss with origin (#498) 2024-04-26 19:39:08 +08:00
nathan 32bdfb7037 chore: update logs 2024-04-26 13:29:57 +08:00
Nathan.fooo 7b4b9d0c10
chore: folder data check (#496)
* chore: folder data check

* chore: stop observe remote change when specific type of collaboration object override with incorrect data

* chore: commit sqlx file
2024-04-26 13:20:06 +08:00
nathan a045713222 chore: update log 2024-04-25 12:19:26 +08:00
Nathan.fooo a0593b5502
chore: refactor return response with object id (#494)
* chore: refactor return response with object id

* chore: fix ci

* chore: fix ci
2024-04-24 12:07:54 +08:00
Nathan.fooo 5f970a625a
chore: use rust 1.77 (#493) 2024-04-24 10:47:18 +08:00
Nathan.fooo 1019b8c5b6
chore: update collab rev (#490) 2024-04-22 17:15:12 +08:00
Nathan.fooo efbe9c4a93
chore: move collab rt crate (#489)
* chore: create appflowy collaborate crate

* chore: remove collab-rt

* chore: fmt

* chore: main
2024-04-22 12:14:41 +08:00
Nathan.fooo 123484c864
chore: sync missing update from vector (#481) 2024-04-20 20:50:16 +08:00
Nathan.fooo 4168d72390
chore: using redis conn for each action (#476)
* chore: batch insert

* chore: opti redis conn

* chore: fix test
2024-04-17 11:41:50 +08:00
Nathan.fooo 8ab7815740
chore: reduce pool lock (#475)
* chore: reduce pool lock

* chore: clippy
2024-04-16 20:26:26 +08:00
Nathan.fooo 0be4d2d5b5
ci: try to fix flaky test (#473)
* chore: update ping config

* chore: fix test

* chore: fix test
2024-04-16 18:20:36 +08:00
Nathan.fooo f3279e9b4e
feat: implement grpc for history server (#470)
* ci: docker compose for ci

* chore: add test

* chore: imple grpc

* chore: fix test

* chore: fix ci
2024-04-16 14:10:56 +08:00