Commit Graph

265 Commits

Author SHA1 Message Date
Nathan.fooo 14cfaa0bbd
chore: insert metadata after import notion zip file (#871)
* chore: insert metadata after import notion zip file

* chore: update logs
2024-10-10 22:32:02 +08:00
Nathan.fooo 33e63fae5c
chore: try to fix pg lock timeout (#864)
* chore: try to fix pg lock timeout

* chore: add logs for insert collab

* chore: add timeout for duplicate

* chore: timeout for pending write
2024-10-07 16:49:50 +08:00
Nathan.fooo 38ff1874e4
chore: using copy to improve write speed of batch collab api endpoint (#862)
* chore: using copy to improve write speed of batch collab api endpoint

* chore: insert to redis

* chore: fix compile
2024-10-07 08:58:19 +08:00
Nathan.fooo 9c0dffacef
feat: implement import notion zip file api endpoint (#840)
* chore: implement api endpoint

* chore: worker

* chore: async zip

* chore: test bulk insert

* chore: insert collab

* chore: custom task

* chore: consume un acked message

* chore: fix compile

* chore: add test

* test: update

* chore: save import record

* chore: save import record

* chore: fix ci

* chore: remove unused deps

* chore: update docker file

* chore: build worker images

* chore: use small int

* chore: use small int

* chore: rm protoc deps in runtime

* chore: move collab cache to database crate

* chore: update test

* chore: rm health api endpoint

* chore: clippy

* chore: update ci

* chore: add test

* chore: upgrade collab

* chore: clippy

* chore: update test

* chore: use custom host

* chore: config nginx

* chore: install cert
2024-10-03 22:22:29 +08:00
khorshuheng 5b799208a8 chore: remove publish collab related dead code 2024-10-03 11:05:28 +08:00
Khor Shu Heng 11732324b5
feat: support fetching encoded collab in batch from memory (#837) 2024-10-02 10:22:40 +08:00
khorshuheng 81c9042439 feat: send email to workspace owner when user request access 2024-10-01 11:54:55 +08:00
Khor Shu Heng 757d0e5380
fix: incorrect requester avatar url and workspace member count (#847) 2024-09-29 11:03:08 +08:00
khorshuheng 6c6de08eaa fix: access not granted after approval 2024-09-26 18:19:33 +08:00
khorshuheng 813fa29253 feat: add endpoints to allow workspace owner to approve web page view request 2024-09-26 14:59:33 +08:00
Khor Shu Heng 0b193e1606
feat: add endpoint for getting page view collab (#831) 2024-09-18 12:56:34 +08:00
khorshuheng 180db846d9 fix: filter out templates which has not been published 2024-09-16 00:05:22 +08:00
Zack Fu Zi Xiang 4b59574392
chore: add specific error code for wrong invitee 2024-09-13 21:02:09 +08:00
Zack Fu Zi Xiang 11cdc862f2
feat: additional fields for workspace invitation 2024-09-13 17:43:10 +08:00
Zack Fu Zi Xiang f25066f51f
feat: add get invitation by id 2024-09-12 15:10:48 +08:00
Khor Shu Heng abae8d2d1b
feat: Support s3 as published collab storage (#798)
* chore: remove unused published collab methods

* feat: support s3 as published collab storage
2024-09-11 20:52:33 +08:00
Nathan.fooo 07d6001cf8
chore: check before write to hisotry (#809)
* chore: check before write to hisotry

* chore: clippy

* chore: clippy

* chore: calculate edit count

* chore: clippy

* chore: exit loop

* chore: exit loop

* chore: update log
2024-09-10 22:10:59 +08:00
Zack Fu Zi Xiang cf7d4b0ac9
Merge branch 'main' into fix/del-collab-emb 2024-09-05 11:21:06 +08:00
Nathan.fooo 5d356718fa
fix: using spawn_blocking in order to not block the runtime (#793)
* fix: using blocking in order to not block the runtime

* chore: increase deps count
2024-09-05 09:19:34 +08:00
Zack Fu Zi Xiang 312411cca2
fix: slow query deleting collab embeddings 2024-09-05 00:06:04 +08:00
Nathan.fooo 4e384b982e
chore: pass chat metadata (#789)
* chore: pass chat metadata

* chore: clippy
2024-09-04 17:19:39 +08:00
Zack Fu Zi Xiang 584ed0863f
chore: fmt sqlx 2024-09-04 12:03:25 +08:00
Zack Fu Zi Xiang 6d0a7cc95b
fix: delete user should only remove owned workspaces 2024-09-04 12:00:34 +08:00
khorshuheng 89f43bb24a chore: replace query with query scalar for retrieving published view id 2024-09-03 15:28:01 +08:00
khorshuheng 56b9f9daf4 feat: published view outline endpoint 2024-09-03 15:27:59 +08:00
Zack Fu Zi Xiang 0fc6a3c4e7
feat: use common table expression to improve user profile sql 2024-09-03 12:08:37 +08:00
Zack 826546c5cb
feat: duplicate published doc (#700)
* chore: test collab sync

* feat: folder view for user workspace

* feat: add private indicator

* chore: use collab cache instead

* chore: initial impl

* chore: cargo sqlx

* fix: write back to collab cache

* fix: assign page id

* fix: text map

* chore: connect api and refactor

* chore: switch to using mem storage

* fix: collab type

* feat: use group manager to manage sync

* feat: try add send command instead

* chore: add client api code

* feat: try use sink and stream from group collab

* chore: disable sync first

* fix: insert page before stopping group

* feat: add extra for view

* feat: add metadata to doc

* fix: icon

* fix: page circular dep

* fix: page circular dep

* fix: live update

* feat: database initial impl

* feat: workspace database

* chore: link database

* fix: database validation

* fix: workspace database oid

* fix: workspace database oid

* fix: specify view layout

* feat: add txn when inserting collab and rework database views

* fix: parent view id for new database

* fix: database view for folder

* fix: database ids

* fix: database row id

* fix: main database layout and write database before folder

* fix: metadata for non database main view

* chore: database in doc wip

* chore: filter visible views

* chore: use structure value instead of json

* chore: database in doc wip

* fix: remove child item push

* fix: main database view meta

* chore: rework inline database in doc

* fix: make duplicate more error forgiving

* chore: use get instead of get key value

* chore: merge with main

* feat: set max depth for folder view

* chore: update dependency counter

* chore: remove unneeded arg

* chore: simplify collab new from source

* chore: remove unneeded param

* chore: rename and doc

* feat: rework inline database and use duplicated ref

* chore: simplify params

* chore: fmt

* feat: deduplicate database wip

* fix: compatible with newer appflowy version

* fix: database views

* feat: database in doc

* chore: cargo clippy

* chore: update dockerfile rust toolchain

* fix: doc inline db parent id

* fix: database id

* fix: document data modification

* chore: minor refactor and todo

* chore: remove unneeded nesting

* fix: enforce ordering for views to add

* chore: add doc and test case for folder view

* fix: remove the need for cloning published collab

* fix: some review

* chore: move folder conversion logic

* fix: remove collab folder deps on shared entity

* chore: cargo fmt

* fix: compile

* chore: remove group manager dependency

* fix: dep count

* fix: add messages sent by server

* fix: dep count

* chore: add test for doc ref and inline doc database

* chore: cargo clippy

* chore: add more test scenarios

* chore: fix tests

* chore: get database id

* chore: update collab

* chore: add more assert and database row id checks

* fix: suggestions from review

* chore: sqlx

* fix: accumulate collab before insert

* chore: add tokio spawn blocking for encoding

* fix: reduce limit for publish collabs

---------

Co-authored-by: Bartosz Sypytkowski <b.sypytkowski@gmail.com>
Co-authored-by: nathan <nathan@appflowy.io>
2024-09-03 09:12:29 +08:00
Nathan.fooo 365c64c5c2
chore: use colla storage instead of select from db directly (#782) 2024-09-02 21:25:21 +08:00
Khor Shu Heng 52f33e4d85
feat: endpoint for avatar upload (#736) 2024-08-23 16:36:09 +08:00
Khor Shu Heng 9c8e718246
feat: Template CRUD Endpoint (#731)
* feat: template crud endpoint

* fix: clippy error

* fix: categories for related view

* fix: add created at and last updated at to template response

* feat: template api delete endpoint

* feat: include number of template count for template creator

* fix: use params instead of individual fields for template api

* fix: seach template creator by name query

* chore: simplify query

* feat: support template count limit for template homepage
2024-08-20 15:22:56 +08:00
khorshuheng 2f0a705e16 fix: update updated_at fields when updating template creator and category 2024-08-19 13:06:30 +08:00
khorshuheng 0c31191586 fix: improve query indentation for template creator sqlx 2024-08-19 12:54:05 +08:00
khorshuheng 3481dab2bf fix: sorting order of template category and creator 2024-08-16 14:26:04 +08:00
khorshuheng 59e5a13ee4 fix: replace rank with priority 2024-08-14 14:17:21 +08:00
khorshuheng fc56654e61 chore: use UUID value instead of ref for func arg as UUID implements Copy 2024-08-14 13:26:36 +08:00
khorshuheng f6e78a941f feat: api endpoint for template categories and creators 2024-08-12 13:43:14 +08:00
Khor Shu Heng ea27e87103
feat: support avatar url for publish comment and reaction endpoint (#711)
* chore: update sqlx to 0.8

* feat: support avatar url for publish comment and reaction

* chore: simplify reactions and comments ops using query as
2024-08-08 23:58:28 +08:00
Nathan.fooo 0b3949152b
feat: support chat with markdown (#718)
* chore: store metatdata

* chore: support markdown

* chore: update test
2024-08-08 13:19:19 +08:00
Khor Shu Heng 21bbfa828e
chore: update sqlx to 0.8 (#710)
* chore: update sqlx to 0.8

* fix: handle potential encoding error during add args

* chore: more description for SqlxArgEncodingError
2024-08-05 23:44:11 +08:00
Nathan.fooo a371912c61
chore: enable chat with provided context (#713)
* chore: enable chat with provided context

* chore: rename

* chore: update create chat message api endpoint

* chore: use list context

* chore: use list context

* chore: fix test

* chore: update api endpoint

* chore: rename client api function

* chore: rename client api function

* chore: expose entity

* chore: update sqlx files

* chore: update test
2024-08-05 14:06:44 +08:00
Khor Shu Heng ff7de66dfa feat: add can be deleted field to GlobalComment 2024-07-29 19:32:16 +08:00
khorshuheng 230358fd2f fix: rename uid to uuid in AFWebUser struct for clarity 2024-07-29 16:04:54 +08:00
khorshuheng 422683d717 fix: order reactions by reaction type first creation date 2024-07-29 16:02:03 +08:00
khorshuheng b6c9f541e2 fix: delete reaction from comment SQL query should be based on comment id, not view id 2024-07-29 14:47:33 +08:00
khorshuheng ea8ca36b5b fix: share query param struct for api client and server 2024-07-29 11:36:38 +08:00
Khor Shu Heng 62f32e8757 feat: return user struct instead of uuid for reaction 2024-07-27 12:55:07 +08:00
Khor Shu Heng b861f0a703 feat: endpoints for reactions on published view 2024-07-26 22:56:47 +08:00
Khor Shu Heng 122520e9f4 fix: order comments by recency 2024-07-26 16:28:35 +08:00
Zack Fu Zi Xiang fc62a079c0
chore: merge with main 2024-07-25 21:05:29 +08:00
Khor Shu Heng f71ac07ae7 chore: rename GlobalCommentCreator to AFWebUser to allow potential reuse 2024-07-25 14:39:02 +08:00
Khor Shu Heng f535950643 feat: add database operations for publish comment crd 2024-07-25 13:32:43 +08:00
Khor Shu Heng 6eea93d774 feat: add create, retrieve, delete API for publish view global comment 2024-07-25 13:32:43 +08:00
Zack Fu Zi Xiang 11c1521b71
feat: improve workspace api 2024-07-25 04:19:40 +08:00
Bartosz Sypytkowski a8ed5be8d0 fix: cargo fmt 2024-07-15 19:21:10 +02:00
Bartosz Sypytkowski f5ca71e701 chore: remove pg advisory lock 2024-07-15 19:09:26 +02:00
Zack Fu Zi Xiang d06dbadc05
fix: update publish collab 2024-06-29 13:50:56 +08:00
Zack Fu Zi Xiang 395486adc3
fix: fix various publish issues 2024-06-29 11:23:23 +08:00
Bartosz Sypytkowski 71899ec792 chore: disabling indexing on workspace prunes collab embeddings 2024-06-28 09:09:18 +02:00
Bartosz Sypytkowski 2f37f621b2 fix: collab update violating primary key 2024-06-28 07:11:33 +02:00
Bartosz Sypytkowski 9ea085d6db chore: fix cargo formatting 2024-06-28 05:40:44 +02:00
Bartosz Sypytkowski b79fe41af3 chore: moving embeddings into collabparams 2024-06-28 05:39:56 +02:00
Bartosz Sypytkowski b1ebbecc5f chore: moving appflowy indexer into appflowy cloud 2024-06-28 05:39:53 +02:00
Bartosz Sypytkowski b978033c77 fix: make search results visible for members 2024-06-26 13:03:11 +02:00
Bartosz Sypytkowski 6592f00a20 chore: change query to use workspace directly 2024-06-25 08:07:32 +02:00
Bartosz Sypytkowski 2ff2a77465 fix: do not throw errors if indexer found document before workspace was created 2024-06-25 07:30:48 +02:00
Bartosz Sypytkowski ef27a148b9 chore: rename workspace settings disable indexing to disable search indexing in database queries 2024-06-24 10:54:42 +02:00
Zack Fu Zi Xiang 0bf6d3bd60
feat: change doc name to publish name 2024-06-21 11:18:44 +08:00
Zack Fu Zi Xiang 0adbf132be
chore: merge with main 2024-06-20 21:15:25 +08:00
Nathan.fooo a3af38162e
chore: fix upload filem meta key by adding parent dir as prefix (#635)
* chore: fix upload filem meta key by adding parent dir as prefix

* chore: fix test
2024-06-20 21:12:28 +08:00
Zack Fu Zi Xiang b525e333f0
feat: batch unpublish 2024-06-20 12:05:12 +08:00
Zack Fu Zi Xiang 31f5c786a6
chore: merge with main 2024-06-20 11:20:16 +08:00
Zack Fu Zi Xiang d4dd350da1
feat: batch publish item with metadata 2024-06-20 11:15:08 +08:00
Nathan.fooo 30f87457f3
feat: support uploading file with multiple part (#625)
* refactor: replace crate

* chore: remove s3-rust crate

* chore: test multiple upload

* chore: client api

* chore: client api

* chore: v1 version upload,download

* chore: fix test

* chore: fix test

* chore: remove workspace test

* chore: fix wasm build

* chore: update path

* chore: update path

* chore: rename

* chore: expose file entity

* chore: introduce client-api-entity

* chore: fix build

* chore: new url

* chore: fix test

* chore: merge main

* chore: fix clippy
2024-06-19 13:06:58 +08:00
Zack Fu Zi Xiang 6c09839728
feat: batch metadata 2024-06-19 01:30:51 +08:00
Zack Fu Zi Xiang 09828c5eea
feat: add impl for get collab info 2024-06-18 19:07:48 +08:00
Zack Fu Zi Xiang 18e9d0abae
feat: use view id 2024-06-18 18:20:05 +08:00
Bartosz Sypytkowski 9ea4de06fd
Merge pull request #630 from AppFlowy-IO/track-ai-responses
chore: change open ai token tracking to daily basis
2024-06-18 10:04:04 +02:00
Bartosz Sypytkowski 9ca904f943 chore: change open ai token tracking to daily basis 2024-06-18 06:53:39 +02:00
Zack Fu Zi Xiang d2cb0d4486
chore: remove unneeded api 2024-06-17 21:33:36 +08:00
Zack Fu Zi Xiang cb74f38ac8
fix: json metadata 2024-06-16 20:41:50 +08:00
Zack Fu Zi Xiang 05b284f7e7
feat: client code for namespace changes 2024-06-16 14:06:04 +08:00
Zack Fu Zi Xiang 233b757698
feat: add published endpoints 2024-06-15 21:47:11 +08:00
Zack Fu Zi Xiang 2ab59e79a3
feat: get published collab meta data 2024-06-15 21:20:59 +08:00
Zack Fu Zi Xiang de413844dc
feat: put and get published collab data 2024-06-15 14:59:18 +08:00
Zack Fu Zi Xiang 94bd62869c
feat: delete publish collab 2024-06-15 11:51:05 +08:00
Zack Fu Zi Xiang 18da7b873b
feat: publish namespace and metadata 2024-06-14 21:35:39 +08:00
Bartosz Sypytkowski b52369de7f
feat: add workspace settings with disable indexing option (#606)
* feat: add workspace settings with disable indexing option

* chore: added api workspace settings endpoint

* chore: add workspace settings read and write to client api

* chore: add tests for workspace settings client api

* fix: enforce authorization rules on workspace settings api

* chore: fix appflowy indexer test configuration

* chore: add dotenvy to appflowy indexer tests

* chore: override appflowy indexer env vars for tests

* chore: add appflowy indexer preindex env var
2024-06-06 09:26:21 +08:00
Bartosz Sypytkowski 58e01f4e8d
fix: index documents with missing embeddings at service start (#597)
* feat: appflowy-indexer indexes documents with missing embeddings at start

* chore: add tests to check initial indexing of collabs

* chore: log user search requests and response

* fix: wrong appflowy openai api key env var name

* chore: fill openai keys for integration testing

* chore: silence flaky test
2024-06-04 16:50:06 +08:00
Nathan.fooo b36715dc24
chore: generate answer manually (#594)
* chore: generate answer manually

* chore: rename

* chore: return reply message id

* chore: save message

* chore: commit schema files
2024-06-03 08:06:23 +08:00
Nathan.fooo 1cc5b58254
chore: update client api (#593) 2024-06-02 20:20:14 +08:00
Nathan.fooo edfcb5c1ea
chore: extend chat message with meta (#592) 2024-06-01 19:32:39 +08:00
Nathan.fooo 1537c4d1f6
chore: update chat test (#590) 2024-05-30 17:15:13 +08:00
Bartosz Sypytkowski c4702bbbdf
feat: Workspace token usage (#584)
* feat: register open ai token usage during indexing

* feat: register open ai token usage during search

* chore: fixed open ai token usage when searching for documents
2024-05-29 16:07:56 +08:00
Bartosz Sypytkowski d3e59fa801
feat: Appflowy-indexer service and /api/search endpoint (#582)
* feat: implement appflowy-indexer and search api endpoint

* chore: store embedding content type

* chore: more descriptive messages for open ai missing key

* chore: fix sqlx errors in tests

* chore: document watcher - change plain text rendering to use depth-first scan

* chore: added nested document blocks example
2024-05-28 21:09:29 +08:00
Khor Shu Heng 93b4a1516c
feat: add postgres listeners for collab service (#575) 2024-05-27 09:12:32 +08:00
Nathan.fooo 559d924cd1
feat: stream chat message (#577)
* chore: save author id

* chore: stream response

* chore: stream chat message
2024-05-26 22:44:08 +08:00
Nathan.fooo ae3e075475
chore: Integrate with client-api, update chat message (#574)
* chore: update

* chore: get chat message with next back option

* chore: update next back

* chore: return answer

* chore: return answer

* chore: return answer

* chore: update client api

* chore: fix ci
2024-05-24 15:04:50 +08:00
Nathan.fooo 4eb1e6bceb
chore: generate history using redis steam (#570)
* chore: combine test

* chore: add snapshot test

* chore: add test
2024-05-23 07:57:20 +08:00
Nathan.fooo 05e7f1cda7
Integrate ai chat (#569)
* chore: fix compile

* chore: implement client api

* chore: implement client api

* chore: clippy
2024-05-22 11:18:07 +08:00
Zack ec7eb54bfc
chore: Self host improvement (#561)
* fix: remove multiple ways to specify database name

* feat: resent email for already invited user

* feat: mailer address from smtp username

* feat: allow user defined smtp port
2024-05-18 08:33:08 +08:00