Commit Graph

26 Commits

Author SHA1 Message Date
Nathan.fooo 1131818eb7
fix: support index document in the background, fix stack overflow when calling rayon::spawn (#1099)
* chore: batch index

* chore: format log

* chore: index workspace

* chore: fix stack overflow

* chore: background index

* chore: clippy

* chore: filter tasks

* chore: clippy

* chore: add metrics

* chore: fix test
2024-12-24 14:30:17 +08:00
Nathan.fooo ec9c38254b
chore: open ai call directly (#1074)
* chore: use open ai embedding

* chore: update test

* chore: retry

* chore: metrics

* chore: clippy

* chore: env

* chore: delete test
2024-12-15 23:29:46 +08:00
Bartosz Sypytkowski 9ff6f1c744
S3 collab cache (#1028)
* chore: collab cache for S3

* chore: adjust disk cache api to accomodate s3

* chore: move postgres dependent ops to disk cache

* chore: replace blob inserts from pg to s3

* chore: delete blob and collab exist now use s3

* chore: fix clippy erorrs

* chore: post rebase fixes

* chore: fix clippy warnings

* chore: fix imports

* chore: make snapshots work over S3

* chore: remove dead code

* chore: use compressed snapshots

* chore: add zstd compression

* chore: introduce collab size threshold to keep smaller collabs in postgres

* chore: remove collabs from S3 if they were put to postgres

* chore: update tests
2024-12-03 13:08:55 +08:00
Nathan.fooo 655f13bc27
chore: search and chat (#999)
* chore: add test for search and chat

* chore: update test

* chore: update test

* chore: update ci

* chore: fix security audio

* chore: multiple core docker build

* chore: multiple core docker build

* chore: update ci

* chore: update model setting

* chore: test ci

* chore: use tiktoken to calcualte token length

* chore: remove env

* chore: use spawn_blocking with condition

* chore: docs

* chore: clippy

* chore: clippy

* chore: docker logs

* chore: pass message id

* chore: clippy
2024-11-16 14:52:12 +08:00
Nathan.fooo 38ff1874e4
chore: using copy to improve write speed of batch collab api endpoint (#862)
* chore: using copy to improve write speed of batch collab api endpoint

* chore: insert to redis

* chore: fix compile
2024-10-07 08:58:19 +08:00
Zack Fu Zi Xiang 922418b122
chore: merge with main 2024-09-12 10:09:42 +08:00
Zack Fu Zi Xiang cff144a857
chore: add test for duplicating doc in database row 2024-09-11 16:38:24 +08:00
Nathan.fooo 7cd88b7502
fix: batch create collab api endpoint might cause tokio runtime issue (#810)
* fix: batch create collab api endpoint might cause tokio runtime issue

* chore: remove test

* chore: clippy
2024-09-11 11:32:42 +08:00
Zack Fu Zi Xiang d1635cb99c
chore: add test 2024-09-06 15:30:28 +08:00
Nathan.fooo 5d356718fa
fix: using spawn_blocking in order to not block the runtime (#793)
* fix: using blocking in order to not block the runtime

* chore: increase deps count
2024-09-05 09:19:34 +08:00
Bartosz Sypytkowski a706c312b3 chore: making embeddings code compile 2024-06-28 05:40:41 +02:00
Nathan.fooo d994b10115
chore: appflowy ai client (#547)
* chore: appflowy ai client

* chore: clippy

* chore: clippy

* chore: fix ci

* chore: disable index search
2024-05-12 09:23:32 +08:00
Nathan.fooo 7b4b9d0c10
chore: folder data check (#496)
* chore: folder data check

* chore: stop observe remote change when specific type of collaboration object override with incorrect data

* chore: commit sqlx file
2024-04-26 13:20:06 +08:00
Nathan.fooo a0593b5502
chore: refactor return response with object id (#494)
* chore: refactor return response with object id

* chore: fix ci

* chore: fix ci
2024-04-24 12:07:54 +08:00
Nathan.fooo 3901356e8a
chore: bump collab version (#471) 2024-04-15 12:35:14 +08:00
Nathan.fooo 01f1af98f5
chore: wrapper for collab (#448)
* chore: wrapper for collab

* chore: add tests

* chore: fix audit

* chore: init sync

* chore: disable wasm user test

* chore: disable encode state when start

* chore: fix test
2024-04-05 11:30:00 +08:00
Nathan.fooo c77329ea11
chore: encode collab with validate check (#439)
* chore: bump collab

* chore: fix test

* chore: fix test
2024-04-03 09:07:35 +08:00
Nathan.fooo d77fb77a3a
feat: validate collab data before writing to disk (#425)
* chore: validate collab

* chore: update test

* chore: partition key

* chore: save collab

* chore: save collab
2024-03-29 14:55:51 +08:00
Nathan.fooo deffd074b9
chore: check collab before insert (#349)
* chore: check collab before insert

* chore: fix audit

* chore: speed test

* chore: add expirate duration for cache encode collab

* chore: fix test
2024-02-25 12:38:07 +08:00
Nathan.fooo d851b303bf
refactor: async send ws message in http request (#296)
* refactor: async send ws message in http request

* chore: update

* chore: update logs

* chore: update logs

* chore: clippy

* chore: add device id in header

* chore: fix test

* chore: fix device_id

* chore: fix client id

* chore: ratelimt for client

* chore: bump collab
2024-02-06 15:21:48 +08:00
Nathan.fooo 3377e664d5
feat: create websocket crate for wasm and native (#269)
* feat: create websocket crate for wasm and native

* chore: update

* chore: replace with websocket::message

* chore: fix client api build

* chore: fix compile

* fix: ci

* refactor: add client api test crate
2024-01-23 14:25:01 +08:00
Nathan.fooo 79702b1ced
feat: stream create list of collab (#253) 2024-01-11 09:28:28 +08:00
Nathan.fooo 284e2bf638
chore: compress collab data (#249)
* chore: compress collab data

* chore: custom buffer size

* chore: compression ws message

* chore: update log

* chore: update log

* chore: update client max body size

* chore: update client api

* docs: update docs

* chore: retry transaction

* chore: update timeout

* chore: stop previous loop

* chore: log ping error

* chore: log send error

* chore: log error

* chore: update log

* chore: async compress

* feat: stream payload

* chore: update load

* chore: update

* chore: update log

* chore: fix cmt

* fix: client message from http

* chore: bump collab
2024-01-10 08:05:57 +08:00
Nathan.fooo 6d0612ffaf
feat: encode batch create collab data as binary (#242) 2024-01-03 06:07:21 +08:00
Nathan.fooo 4d113425fe
feat: override collab when create (#234)
* feat: override collab when create

* chore: clippy

* chore: fmt
2023-12-25 09:40:32 +08:00
Nathan.fooo a07cc0b903
feat: Insert multiple collab (#233)
* chore: remove panic hook

* chore: support insert multiple collabs
2023-12-25 02:03:41 +08:00