c4514ca5bf
chore: add explanation
2024-12-10 10:12:33 +01:00
9971c23749
fix: update redis cache when inserting collabs in batches
2024-12-10 10:07:54 +01:00
813179fdb0
chore: Remove async awaits from RealtimeServerActor ( #1059 )
...
* chore: reduce number of async await methods in connection manager
* chore: remove async handlers from rt actor
2024-12-10 15:20:28 +08:00
29cb70a362
fix: synchronization issue due to incorrect collab origin ( #1054 )
2024-12-09 14:19:00 +08:00
a598aed5ec
chore: remove empty map when query is empty ( #1050 )
2024-12-07 10:20:53 +08:00
66deaf3fa3
chore: move collab cache for appflowy collaborate service and add metrics ( #1045 )
...
* chore: move collab cache for appflowy collaborate service and add metrics
* chore: add histogram for measuring time spent in postgres transactions
* chore: delete blobs in chunks
* chore: bring back redis cache filling by import worker
* chore: cargo fmt
* chore: make import workspace update existing collabs
* chore: bulk insert collab deduplicate records
2024-12-06 19:55:51 +08:00
ee87bb9bd6
chore: setup tokio console ( #1046 )
...
* chore: setup tokio console
* chore: setup tokio console
* chore: clippy
* chore: clippy
2024-12-06 11:33:04 +08:00
f79042fcc5
Merge pull request #1043 from AppFlowy-IO/web-update-fix
...
fix: web update should update both disk and mem cache
2024-12-06 10:44:52 +08:00
afcd1130c3
feat: chat settings ( #1044 )
...
* feat: chat settings
* chore: fix sqlx
2024-12-05 23:30:11 +08:00
aa64b7cd6b
fix: web update should update both disk and mem cache
2024-12-05 16:51:56 +08:00
b2f8a95ec4
chore: setup new collab in the db
2024-12-05 06:29:19 +01:00
5e7b5f87fa
chore: merge branch main into stateless
2024-12-03 09:10:01 +01:00
51bf9a85c7
chore: make collab storage switch between s3 and postgres configurable ( #1035 )
2024-12-03 14:09:09 +08:00
9ff6f1c744
S3 collab cache ( #1028 )
...
* chore: collab cache for S3
* chore: adjust disk cache api to accomodate s3
* chore: move postgres dependent ops to disk cache
* chore: replace blob inserts from pg to s3
* chore: delete blob and collab exist now use s3
* chore: fix clippy erorrs
* chore: post rebase fixes
* chore: fix clippy warnings
* chore: fix imports
* chore: make snapshots work over S3
* chore: remove dead code
* chore: use compressed snapshots
* chore: add zstd compression
* chore: introduce collab size threshold to keep smaller collabs in postgres
* chore: remove collabs from S3 if they were put to postgres
* chore: update tests
2024-12-03 13:08:55 +08:00
3e3bf693fd
chore: upload api endpoint for web upload ( #1024 )
...
* chore: upload api endpoint for web upload
* chore: use rust 1.78
* fix: test
2024-11-28 09:09:09 +08:00
4fb6df5c14
chore: reduce number of SQL statements executed when inserting/updating collab ( #1022 )
...
* chore: reduce number of SQL statements executed when inserting new collab
* chore: add workspace id check
* chore: fix previous query
* chore: upsert collabs and embddings as stored procedures
* chore: fix clippy errors
* chore: fix stored procedure to compare workspaces correctly
2024-11-27 13:00:00 +08:00
0ec12c5f2f
chore: add file size ( #1015 )
2024-11-22 16:31:50 +08:00
d6317813d7
chore: allow dead code
2024-11-20 09:06:00 +01:00
b91a5fac8e
chore: post merge fixes
2024-11-20 08:56:16 +01:00
d0f9059f3f
Merge branch 'main' into stateless
2024-11-20 08:36:28 +01:00
1e18180e9d
chore: set max import zip file size ( #1011 )
...
* chore: set max import zip file size
* chore: fix test
2024-11-20 14:07:36 +08:00
fd98351473
chore: adjust redis env ( #1003 )
2024-11-17 14:26:59 +08:00
3799966f12
chore: store pending collab writes in memory ( #1000 )
...
* chore: write immediatelly actually writes immediatelly
* chore: fix clippy errors
* chore: add metrics to new storage queue impl
* chore: set collab batch write capacity to the same as on main branch
2024-11-17 13:25:42 +08:00
d798c81ba4
chore: support split by text len ( #1002 )
...
* chore: support split by text len
* chore: update docs
* chore: update tests
2024-11-17 12:45:20 +08:00
dcbc84dacc
chore: recreate group if it isn't exist ( #1001 )
2024-11-17 12:14:50 +08:00
655f13bc27
chore: search and chat ( #999 )
...
* chore: add test for search and chat
* chore: update test
* chore: update test
* chore: update ci
* chore: fix security audio
* chore: multiple core docker build
* chore: multiple core docker build
* chore: update ci
* chore: update model setting
* chore: test ci
* chore: use tiktoken to calcualte token length
* chore: remove env
* chore: use spawn_blocking with condition
* chore: docs
* chore: clippy
* chore: clippy
* chore: docker logs
* chore: pass message id
* chore: clippy
2024-11-16 14:52:12 +08:00
06b7ce0a29
chore: post merge fixes
2024-11-16 07:33:18 +01:00
401cc3ad25
Merge branch 'main' into stateless
2024-11-16 07:22:34 +01:00
d0c212ff10
refactor: remove enforcer group ( #948 )
2024-11-15 14:08:12 +08:00
b10a0e7829
chore: fix typo
2024-11-15 13:45:25 +08:00
97f9ff3dd8
fix: byte index 8000 is not a char boundary ( #995 )
...
* chore: fix split text boundary error and add related tests
* chore: reduce clone
2024-11-15 13:38:08 +08:00
d8075a9368
Merge pull request #994 from AppFlowy-IO/fix/smtp-email-username
...
feat: separate smtp username from email
2024-11-14 22:00:43 +08:00
4b2eb1b1ba
feat: separate smtp username from email
2024-11-14 16:00:15 +08:00
06a6ea2985
chore: retry upload file when get s3 service error ( #993 )
...
* chore: retry upload file when get s3 service error
* chore: ingore upload payload error
2024-11-14 14:39:36 +08:00
f6fef9918b
chore: split document content into multiple embeddings ( #990 )
...
* chore: split document content into multiple embeddings
* chore: fallback to uuid v4 generation due to unmaintained transient dependency
* chore: fix lint errors
2024-11-14 14:15:52 +08:00
50d519c4ea
fix: upload file zip file expire ( #987 )
2024-11-13 14:57:27 +08:00
91b0c50722
chore: remove logs ( #986 )
2024-11-13 13:59:23 +08:00
9778843746
chore: disable history service ( #985 )
...
* chore: disable history service
* chore: fix build
2024-11-13 10:46:15 +08:00
14adf250a8
chore: update ai api endpoint & rename structs ( #953 )
...
* chore: revmap loader type
* chore: revamp context
* chore: fix test
* chore: remove unused handler
* chore: create new chat api endpoint
* chore: add docs
* chore: clippy
* chore: fix test
2024-11-11 20:10:46 +08:00
eb54947ab7
chore: add config option for redis stream group workers count
2024-11-08 04:39:44 +01:00
b9ca480143
chore: add retry limit to which collab snapshot can be made
2024-11-07 13:12:17 +01:00
bc49d73b40
chore: split redis stream readers to active and one shot
2024-11-07 11:43:19 +01:00
2d9a024554
chore: use redis stream router
2024-11-07 07:18:20 +01:00
f3cd597832
chore: fix clippy errors
2024-11-06 10:57:32 +01:00
3fc49a8b21
chore: turn off collab preindexing
2024-11-06 10:10:55 +01:00
863e405263
chore: add more precise logging on get unindexed collabs
2024-11-06 08:04:07 +01:00
99203cc7f3
chore: fix flaky tests
2024-11-05 07:48:14 +01:00
25f0461389
chore: added metrics for new stateless operations
2024-11-05 07:48:14 +01:00
e86c9de316
chore: when saving collab snapshot, do not load it unless there are pending updates in redis
2024-11-05 07:48:14 +01:00
961c482995
chore: prevent hot loops in collab updates stream
2024-11-05 07:48:14 +01:00
2511bd720d
chore: fix clippy errors
2024-11-05 07:48:14 +01:00
ce793fa03e
chore: make collab group prune grace period configurable
2024-11-05 07:48:14 +01:00
395424bfd2
chore: detect and re-request missing updates on the server side
2024-11-05 07:48:14 +01:00
a99f986270
chore: removed appflowy history
2024-11-05 07:48:12 +01:00
81dc66fa7b
chore: fix clippy warnings
2024-11-05 07:47:17 +01:00
f684d3623d
chore: fixed collab ack
2024-11-05 07:47:17 +01:00
fc1a7d28e2
chore: better stress test scenario
2024-11-05 07:47:17 +01:00
de4016659e
chore: add break in cancellation of snapshot task
2024-11-05 07:47:17 +01:00
d624cdf49b
chore: turn off separate appflowy history process
2024-11-05 07:47:17 +01:00
a2d3d7feb5
chore: add perf tests
2024-11-05 07:47:17 +01:00
279a599b66
chore: fix early return from redis collab update stream reader
2024-11-05 07:47:17 +01:00
85e25f887c
chore: don't try to prune empty redis stream
2024-11-05 07:47:17 +01:00
d0b7c7d8e3
chore: update stream - live and current queries
2024-11-05 07:47:17 +01:00
61820d7226
chore: more traces in collab group
2024-11-05 07:47:17 +01:00
a1546909c3
chore: remove server sync protocol (not used anymore)
2024-11-05 07:47:17 +01:00
00f5a5bb7e
chore: code cleanup
2024-11-05 07:47:17 +01:00
3e612e2db6
chore: create method for receiving the most recent snapshot
2024-11-05 07:47:17 +01:00
9c674d5777
chore: changes to make appflowy cloud compile under the latest appflowy collab
2024-11-05 07:47:17 +01:00
bee7caedac
chore: attach indexer to collab persister
2024-11-05 07:47:17 +01:00
97335a94ec
chore: basics of snapshots (unoptimized)
2024-11-05 07:47:17 +01:00
253bb97825
chore: a dirty draft of snapshot saving
2024-11-05 07:47:17 +01:00
9164c494b1
chore: redis stream for awareness
2024-11-05 07:47:17 +01:00
d26d0ff1a7
chore: collab streams
2024-11-05 07:47:17 +01:00
0d6b595ee8
chore: create collab update sink and stream
2024-11-05 07:47:17 +01:00
fe3611cc88
chore: remove broadcast and introduce lease
2024-11-05 07:47:17 +01:00
d3d3378b86
chore: draft of stateless collab group
2024-11-05 07:47:17 +01:00
6aed9609b9
chore: removed realtime shared state tests
2024-11-04 09:21:10 +01:00
356cf5d132
chore: fix clippy errors
2024-11-04 08:29:32 +01:00
f105c2dbc2
chore: fix clippy errors
2024-11-04 08:20:56 +01:00
f9397eaaf2
chore: remove realtime shared state
2024-11-04 07:58:16 +01:00
a5a5a6b59f
fix: import notion file base cases ( #957 )
2024-11-02 16:50:51 +08:00
9629d4cefa
chore: add limit for importing zip file ( #938 )
...
* chore: add limitation for import zip file
* chore: support upload big file
* chore: implement client api
* chore: implement client api
* chore: implement client api
* chore: update logs
* chore: check file size
* chore: last process at
* chore: set content type
* chore: fix test
* chore: try test
* chore: temporary disable test
2024-10-28 08:51:34 +08:00
246cbd3ec4
chore: update nginx config for import ( #935 )
2024-10-25 16:47:29 +08:00
e738508d79
fix: Support notion database row document ( #929 )
...
* chore: fix import notion database
2024-10-24 19:04:06 +08:00
16fb9584f9
fix: masked postgres password in logs
2024-10-24 00:34:31 +08:00
9eb1c36c92
chore: access control interface to return error instead of boolean ( #914 )
2024-10-21 20:20:14 +08:00
57c44818e2
feat: Import metrics ( #911 )
...
* chore: implement metrics for import
* chore: add metrics
2024-10-20 11:05:22 +08:00
b1f37dbbf6
chore: fix petential unzip issues ( #910 )
2024-10-20 10:48:09 +08:00
773f17e524
fix: import zip without root dir ( #898 )
...
* fix: import notion zip file that do not have root dir
* fix: compile
2024-10-18 00:51:53 +08:00
ffee47d0f7
chore: remove workspace and collab listener ( #893 )
2024-10-17 17:43:00 +08:00
3623d9f296
chore: check file md5 before import ( #895 )
2024-10-17 17:18:07 +08:00
8ab5da4aef
fix: limit concurrency level of batch get encode collab ( #892 )
2024-10-17 13:29:27 +08:00
9942e68089
chore: imported workspace should not become the latest visting workspace ( #890 )
...
* chore: imported workspace should not become the latest visting workspace
* chore: sqlx files
* chore: update test
* chore: fmt
2024-10-16 21:02:05 +08:00
d3e4a6870d
chore: template ( #889 )
...
* chore: template
* fix: auto del file lifetime
* chore: delete file if fail to write to disk
* fix: modified html
* chore: set content type
* chore: md5 content
* chore: clippy
* fix: image loaded failed
* fix: build
* chore: bump collab
---------
Co-authored-by: Kilu <lu@appflowy.io>
2024-10-16 17:39:24 +08:00
ec124bcb46
Merge pull request #885 from AppFlowy-IO/additional-ac-config
...
feat: additional access control config
2024-10-16 17:00:54 +08:00
383629ab1c
chore: simplify not enough permission error
2024-10-16 15:10:31 +08:00
d89cbe1c8c
chore: support streaming unzip using env ( #888 )
2024-10-16 14:02:03 +08:00
3c02fa253f
chore: retry import ( #887 )
...
* chore: retry import
* chore: delete temp files
* chore: bump collab
2024-10-16 12:55:53 +08:00
2b02a8d12b
chore: bump collab ( #884 )
2024-10-15 15:56:14 +08:00
1616ceaf4f
chore: use workspace deps ( #880 )
2024-10-14 16:02:22 +08:00
Bartosz Sypytkowski
Khor Shu Heng
Nathan.fooo
khorshuheng
nathan
Zack