Commit Graph

10 Commits

Author SHA1 Message Date
Bartosz Sypytkowski 66deaf3fa3
chore: move collab cache for appflowy collaborate service and add metrics (#1045)
* chore: move collab cache for appflowy collaborate service and add metrics

* chore: add histogram for measuring time spent in postgres transactions

* chore: delete blobs in chunks

* chore: bring back redis cache filling by import worker

* chore: cargo fmt

* chore: make import workspace update existing collabs

* chore: bulk insert collab deduplicate records
2024-12-06 19:55:51 +08:00
Bartosz Sypytkowski 9ff6f1c744
S3 collab cache (#1028)
* chore: collab cache for S3

* chore: adjust disk cache api to accomodate s3

* chore: move postgres dependent ops to disk cache

* chore: replace blob inserts from pg to s3

* chore: delete blob and collab exist now use s3

* chore: fix clippy erorrs

* chore: post rebase fixes

* chore: fix clippy warnings

* chore: fix imports

* chore: make snapshots work over S3

* chore: remove dead code

* chore: use compressed snapshots

* chore: add zstd compression

* chore: introduce collab size threshold to keep smaller collabs in postgres

* chore: remove collabs from S3 if they were put to postgres

* chore: update tests
2024-12-03 13:08:55 +08:00
Nathan.fooo 3e3bf693fd
chore: upload api endpoint for web upload (#1024)
* chore: upload api endpoint for web upload

* chore: use rust 1.78

* fix: test
2024-11-28 09:09:09 +08:00
Nathan.fooo 06a6ea2985
chore: retry upload file when get s3 service error (#993)
* chore: retry upload file when get s3 service error

* chore: ingore upload payload error
2024-11-14 14:39:36 +08:00
Nathan.fooo 9629d4cefa
chore: add limit for importing zip file (#938)
* chore: add limitation for import zip file

* chore: support upload big file

* chore: implement client api

* chore: implement client api

* chore: implement client api

* chore: update logs

* chore: check file size

* chore: last process at

* chore: set content type

* chore: fix test

* chore: try test

* chore: temporary disable test
2024-10-28 08:51:34 +08:00
Nathan.fooo 9c0dffacef
feat: implement import notion zip file api endpoint (#840)
* chore: implement api endpoint

* chore: worker

* chore: async zip

* chore: test bulk insert

* chore: insert collab

* chore: custom task

* chore: consume un acked message

* chore: fix compile

* chore: add test

* test: update

* chore: save import record

* chore: save import record

* chore: fix ci

* chore: remove unused deps

* chore: update docker file

* chore: build worker images

* chore: use small int

* chore: use small int

* chore: rm protoc deps in runtime

* chore: move collab cache to database crate

* chore: update test

* chore: rm health api endpoint

* chore: clippy

* chore: update ci

* chore: add test

* chore: upgrade collab

* chore: clippy

* chore: update test

* chore: use custom host

* chore: config nginx

* chore: install cert
2024-10-03 22:22:29 +08:00
Khor Shu Heng abae8d2d1b
feat: Support s3 as published collab storage (#798)
* chore: remove unused published collab methods

* feat: support s3 as published collab storage
2024-09-11 20:52:33 +08:00
Khor Shu Heng 52f33e4d85
feat: endpoint for avatar upload (#736) 2024-08-23 16:36:09 +08:00
Nathan.fooo a3af38162e
chore: fix upload filem meta key by adding parent dir as prefix (#635)
* chore: fix upload filem meta key by adding parent dir as prefix

* chore: fix test
2024-06-20 21:12:28 +08:00
Nathan.fooo 30f87457f3
feat: support uploading file with multiple part (#625)
* refactor: replace crate

* chore: remove s3-rust crate

* chore: test multiple upload

* chore: client api

* chore: client api

* chore: v1 version upload,download

* chore: fix test

* chore: fix test

* chore: remove workspace test

* chore: fix wasm build

* chore: update path

* chore: update path

* chore: rename

* chore: expose file entity

* chore: introduce client-api-entity

* chore: fix build

* chore: new url

* chore: fix test

* chore: merge main

* chore: fix clippy
2024-06-19 13:06:58 +08:00