Commit Graph

115 Commits

Author SHA1 Message Date
Nathan.fooo 655f13bc27
chore: search and chat (#999)
* chore: add test for search and chat

* chore: update test

* chore: update test

* chore: update ci

* chore: fix security audio

* chore: multiple core docker build

* chore: multiple core docker build

* chore: update ci

* chore: update model setting

* chore: test ci

* chore: use tiktoken to calcualte token length

* chore: remove env

* chore: use spawn_blocking with condition

* chore: docs

* chore: clippy

* chore: clippy

* chore: docker logs

* chore: pass message id

* chore: clippy
2024-11-16 14:52:12 +08:00
Khor Shu Heng d0c212ff10
refactor: remove enforcer group (#948) 2024-11-15 14:08:12 +08:00
nathan b10a0e7829 chore: fix typo 2024-11-15 13:45:25 +08:00
Nathan.fooo 97f9ff3dd8
fix: byte index 8000 is not a char boundary (#995)
* chore: fix split text boundary error and add related tests

* chore: reduce clone
2024-11-15 13:38:08 +08:00
Bartosz Sypytkowski f6fef9918b
chore: split document content into multiple embeddings (#990)
* chore: split document content into multiple embeddings

* chore: fallback to uuid v4 generation due to unmaintained transient dependency

* chore: fix lint errors
2024-11-14 14:15:52 +08:00
Nathan.fooo 91b0c50722
chore: remove logs (#986) 2024-11-13 13:59:23 +08:00
Nathan.fooo 9778843746
chore: disable history service (#985)
* chore: disable history service

* chore: fix build
2024-11-13 10:46:15 +08:00
Bartosz Sypytkowski 6aed9609b9 chore: removed realtime shared state tests 2024-11-04 09:21:10 +01:00
Bartosz Sypytkowski 356cf5d132 chore: fix clippy errors 2024-11-04 08:29:32 +01:00
Bartosz Sypytkowski f105c2dbc2 chore: fix clippy errors 2024-11-04 08:20:56 +01:00
Bartosz Sypytkowski f9397eaaf2 chore: remove realtime shared state 2024-11-04 07:58:16 +01:00
Khor Shu Heng 9eb1c36c92
chore: access control interface to return error instead of boolean (#914) 2024-10-21 20:20:14 +08:00
Nathan.fooo ffee47d0f7
chore: remove workspace and collab listener (#893) 2024-10-17 17:43:00 +08:00
Khor Shu Heng 8ab5da4aef
fix: limit concurrency level of batch get encode collab (#892) 2024-10-17 13:29:27 +08:00
khorshuheng 383629ab1c chore: simplify not enough permission error 2024-10-16 15:10:31 +08:00
Nathan.fooo 1616ceaf4f
chore: use workspace deps (#880) 2024-10-14 16:02:22 +08:00
Khor Shu Heng 8c14612f98
feat: refactor access control to enable swapping implementation (#859) 2024-10-10 13:04:47 +08:00
Nathan.fooo 33e63fae5c
chore: try to fix pg lock timeout (#864)
* chore: try to fix pg lock timeout

* chore: add logs for insert collab

* chore: add timeout for duplicate

* chore: timeout for pending write
2024-10-07 16:49:50 +08:00
Nathan.fooo 38ff1874e4
chore: using copy to improve write speed of batch collab api endpoint (#862)
* chore: using copy to improve write speed of batch collab api endpoint

* chore: insert to redis

* chore: fix compile
2024-10-07 08:58:19 +08:00
Nathan.fooo 9c0dffacef
feat: implement import notion zip file api endpoint (#840)
* chore: implement api endpoint

* chore: worker

* chore: async zip

* chore: test bulk insert

* chore: insert collab

* chore: custom task

* chore: consume un acked message

* chore: fix compile

* chore: add test

* test: update

* chore: save import record

* chore: save import record

* chore: fix ci

* chore: remove unused deps

* chore: update docker file

* chore: build worker images

* chore: use small int

* chore: use small int

* chore: rm protoc deps in runtime

* chore: move collab cache to database crate

* chore: update test

* chore: rm health api endpoint

* chore: clippy

* chore: update ci

* chore: add test

* chore: upgrade collab

* chore: clippy

* chore: update test

* chore: use custom host

* chore: config nginx

* chore: install cert
2024-10-03 22:22:29 +08:00
Khor Shu Heng 11732324b5
feat: support fetching encoded collab in batch from memory (#837) 2024-10-02 10:22:40 +08:00
Nathan.fooo 4c54481d70
chore: bump collab version (#822)
* chore: bump collab version

* chore: update deps count
2024-09-14 23:59:50 +08:00
Nathan.fooo a4b885e13b
chore: set exp time for collab base on collab type (#820)
* chore: set exp time for collab base on collab type

* chore: fix test
2024-09-13 19:04:56 +08:00
Nathan.fooo 1ccff73e49
chore: bump appflowy collab (#814)
* chore: bump appflowy collab

* chore: fix build
2024-09-12 00:28:26 +08:00
Nathan.fooo fff93e6083
chore: Adjust gen history interval (#812)
* chore: adjust generate history interval

* chore: disable verbose log
2024-09-11 13:45:53 +08:00
Nathan.fooo 07d6001cf8
chore: check before write to hisotry (#809)
* chore: check before write to hisotry

* chore: clippy

* chore: clippy

* chore: calculate edit count

* chore: clippy

* chore: exit loop

* chore: exit loop

* chore: update log
2024-09-10 22:10:59 +08:00
Nathan.fooo 5d356718fa
fix: using spawn_blocking in order to not block the runtime (#793)
* fix: using blocking in order to not block the runtime

* chore: increase deps count
2024-09-05 09:19:34 +08:00
Zack 826546c5cb
feat: duplicate published doc (#700)
* chore: test collab sync

* feat: folder view for user workspace

* feat: add private indicator

* chore: use collab cache instead

* chore: initial impl

* chore: cargo sqlx

* fix: write back to collab cache

* fix: assign page id

* fix: text map

* chore: connect api and refactor

* chore: switch to using mem storage

* fix: collab type

* feat: use group manager to manage sync

* feat: try add send command instead

* chore: add client api code

* feat: try use sink and stream from group collab

* chore: disable sync first

* fix: insert page before stopping group

* feat: add extra for view

* feat: add metadata to doc

* fix: icon

* fix: page circular dep

* fix: page circular dep

* fix: live update

* feat: database initial impl

* feat: workspace database

* chore: link database

* fix: database validation

* fix: workspace database oid

* fix: workspace database oid

* fix: specify view layout

* feat: add txn when inserting collab and rework database views

* fix: parent view id for new database

* fix: database view for folder

* fix: database ids

* fix: database row id

* fix: main database layout and write database before folder

* fix: metadata for non database main view

* chore: database in doc wip

* chore: filter visible views

* chore: use structure value instead of json

* chore: database in doc wip

* fix: remove child item push

* fix: main database view meta

* chore: rework inline database in doc

* fix: make duplicate more error forgiving

* chore: use get instead of get key value

* chore: merge with main

* feat: set max depth for folder view

* chore: update dependency counter

* chore: remove unneeded arg

* chore: simplify collab new from source

* chore: remove unneeded param

* chore: rename and doc

* feat: rework inline database and use duplicated ref

* chore: simplify params

* chore: fmt

* feat: deduplicate database wip

* fix: compatible with newer appflowy version

* fix: database views

* feat: database in doc

* chore: cargo clippy

* chore: update dockerfile rust toolchain

* fix: doc inline db parent id

* fix: database id

* fix: document data modification

* chore: minor refactor and todo

* chore: remove unneeded nesting

* fix: enforce ordering for views to add

* chore: add doc and test case for folder view

* fix: remove the need for cloning published collab

* fix: some review

* chore: move folder conversion logic

* fix: remove collab folder deps on shared entity

* chore: cargo fmt

* fix: compile

* chore: remove group manager dependency

* fix: dep count

* fix: add messages sent by server

* fix: dep count

* chore: add test for doc ref and inline doc database

* chore: cargo clippy

* chore: add more test scenarios

* chore: fix tests

* chore: get database id

* chore: update collab

* chore: add more assert and database row id checks

* fix: suggestions from review

* chore: sqlx

* fix: accumulate collab before insert

* chore: add tokio spawn blocking for encoding

* fix: reduce limit for publish collabs

---------

Co-authored-by: Bartosz Sypytkowski <b.sypytkowski@gmail.com>
Co-authored-by: nathan <nathan@appflowy.io>
2024-09-03 09:12:29 +08:00
Nathan.fooo 365c64c5c2
chore: use colla storage instead of select from db directly (#782) 2024-09-02 21:25:21 +08:00
Nathan.fooo 299680c14a
chore: decode encode collab using spawn blocking (#781) 2024-09-02 20:27:40 +08:00
Nathan.fooo abd96d8b56
chore: add APPFLOWY_COLLAB_REMOVE_BATCH_SIZE env to control num of co… (#779)
* chore: add APPFLOWY_COLLAB_REMOVE_BATCH_SIZE env to control num of collabs to be removed in one tick

* chore: clippy
2024-09-02 11:24:40 +08:00
Nathan.fooo c2a5369313
chore: set chat request timeout (#777)
* chore: set chat request timeout

* chore: clippy
2024-09-01 15:16:11 +08:00
Bartosz Sypytkowski 6972f9c4ab
feat: Measure sync (#772)
* chore: redesign collab sync protocol to enable injection of metric tracking

* chore: track metrics for apply update on the server side

* chore: close locks

* chore: post rebase fixes
2024-08-30 21:48:19 +08:00
Bartosz Sypytkowski f75413566e chore: reconnect realtime metrics 2024-08-30 09:56:51 +02:00
Nathan.fooo 1f9a03a83d
chore: enable lock timeout feature (#771) 2024-08-30 14:24:34 +08:00
Nathan.fooo 3e701a1f8b
chore: remove no require data error log (#766)
* chore: remove no require data error log

* chore: clippy
2024-08-30 13:55:53 +08:00
Bartosz Sypytkowski 5b2b717fe7 chore: remove redundant metrics collector 2024-08-30 05:14:44 +02:00
Nathan.fooo 54b811125e
chore: wrap spawn blocking when decoding collab (#767) 2024-08-29 22:53:33 +08:00
Bartosz Sypytkowski 2af1999375
chore: added locks with timeouts (#765)
* chore: added locks with timeouts

* chore: roll back collab locks in client api
2024-08-29 16:13:27 +08:00
Lucas.Xu e7bf221ff0
feat: update built-in README for new users (#757)
* feat: add getting_started, desktop_guide and mobile_guide json

* feat: update built-in readme data

* feat: support adding database as template

* feat: support creating nested documents

* chore: refactor code

* chore: refactor code

* chore: remove println

* chore: update collab version

* feat: update built-in readme and image ref

* chore: remove template icon

* feat: use database::create_with_view to create collab

* chore: add template tests

* chore: update collab version

* chore: remove debug code

* chore: add comment

* chore: fix lint & test

* fix: code review

* fix: rust lint

* chore: update comment

* test: add default workspace test

* test: add default user workspace test

* chore: use workspace database body

* chore: fix typo and clippy

* fix: insert_folder_data_success test

* fix: insert_folder_data_success test

* feat: update content in gettting_started

* chore: update to-dos json

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-08-28 19:40:23 +08:00
Bartosz Sypytkowski 8beac5c85f
chore: reuse group collab for indexing (#737)
* chore: reuse group collab for indexing

* chore: fix linting errors

* chore: post rebase fixes
2024-08-28 00:13:41 +08:00
Nathan.fooo ab14568bbf
chore: use env to use control multiple thread runtime or not (#760) 2024-08-28 00:13:24 +08:00
nathan fee2c37aa4 chore: disable multiple thread in group 2024-08-27 18:12:11 +08:00
Nathan.fooo c8ee6b43ee
chore: try to fix inifinite sync (#756) 2024-08-27 16:41:19 +08:00
Nathan.fooo a21c55cbcc
chore: modify log level for some cases (#754)
* chore: modify log level for some cases

* chore: update logs
2024-08-27 14:57:00 +08:00
Khor Shu Heng 045b6f09cb
feat: add metrics for number of editing users (#730) 2024-08-27 11:56:07 +08:00
Bartosz Sypytkowski 15b2e81579
chore: enable setting up periodic sync step 1 sends from sync plugin (#738)
* chore: enable setting up periodic sync step 1 sends from sync plugin

* chore: rollback to server sending sync step 1 to response to the client

* chore: increase sync step 1 interval in tests
2024-08-27 11:55:27 +08:00
Nathan.fooo cbb9adc29d
chore: enable multiple threads (#753) 2024-08-27 11:15:54 +08:00
Khor Shu Heng c182ad7099
Merge pull request #750 from AppFlowy-IO/ws-client-min-version
add minimum client version validation in websocket connection establishment
2024-08-27 01:35:20 +08:00
Nathan.fooo 8cc410ce65
chore: revamp get collab api request (#752)
* chore: stop retry if record is not found
2024-08-26 23:41:06 +08:00