Commit Graph

362 Commits

Author SHA1 Message Date
Zack Fu Zi Xiang 31f5c786a6
chore: merge with main 2024-06-20 11:20:16 +08:00
Zack Fu Zi Xiang d4dd350da1
feat: batch publish item with metadata 2024-06-20 11:15:08 +08:00
Nathan.fooo 30f87457f3
feat: support uploading file with multiple part (#625)
* refactor: replace crate

* chore: remove s3-rust crate

* chore: test multiple upload

* chore: client api

* chore: client api

* chore: v1 version upload,download

* chore: fix test

* chore: fix test

* chore: remove workspace test

* chore: fix wasm build

* chore: update path

* chore: update path

* chore: rename

* chore: expose file entity

* chore: introduce client-api-entity

* chore: fix build

* chore: new url

* chore: fix test

* chore: merge main

* chore: fix clippy
2024-06-19 13:06:58 +08:00
Zack Fu Zi Xiang 6c09839728
feat: batch metadata 2024-06-19 01:30:51 +08:00
Zack Fu Zi Xiang 09828c5eea
feat: add impl for get collab info 2024-06-18 19:07:48 +08:00
Zack Fu Zi Xiang 18e9d0abae
feat: use view id 2024-06-18 18:20:05 +08:00
Bartosz Sypytkowski 0e50650aa8 chore: use appflowy ai client for embeddings in search api 2024-06-18 10:51:11 +02:00
Zack Fu Zi Xiang 236a22ac97
fix: get workspace namespace 2024-06-18 12:32:01 +08:00
Zack Fu Zi Xiang d2cb0d4486
chore: remove unneeded api 2024-06-17 21:33:36 +08:00
Zack Fu Zi Xiang ae29bbe9fe
fix: published doc alias 2024-06-17 01:07:40 +08:00
Zack Fu Zi Xiang cb74f38ac8
fix: json metadata 2024-06-16 20:41:50 +08:00
Zack Fu Zi Xiang 05b284f7e7
feat: client code for namespace changes 2024-06-16 14:06:04 +08:00
Zack Fu Zi Xiang 233b757698
feat: add published endpoints 2024-06-15 21:47:11 +08:00
Zack Fu Zi Xiang 2ab59e79a3
feat: get published collab meta data 2024-06-15 21:20:59 +08:00
Zack Fu Zi Xiang de413844dc
feat: put and get published collab data 2024-06-15 14:59:18 +08:00
Zack Fu Zi Xiang 94bd62869c
feat: delete publish collab 2024-06-15 11:51:05 +08:00
Zack Fu Zi Xiang 2c7cdefb83
Merge branch 'main' into feat/publish-document 2024-06-14 21:36:17 +08:00
Zack Fu Zi Xiang 18da7b873b
feat: publish namespace and metadata 2024-06-14 21:35:39 +08:00
Nathan.fooo 430e3e15c9
fix: chat stream (#620)
* chore: fix chat stream
2024-06-13 22:46:49 +08:00
Bartosz Sypytkowski d02c7e4ea7 feat: add workspace label to search open ai token metrics 2024-06-12 08:27:37 +02:00
Bartosz Sypytkowski 0b93190cbd chore: let prometheus track open ai tokens used on search 2024-06-12 08:20:57 +02:00
Nathan.fooo ff4384fbd0
feat: translate api endpoint (#612)
* chore: support translate

* chore: support translate

* chore: update endpoint structs
2024-06-12 12:37:11 +08:00
Nathan.fooo d0467e7e2e
feat: Stream chat message (#610)
* chore: test stream

* chore: fix test

* chore: fix test

* chore: add log

* chore: fix parsing

* chore: fix parsing

* chore: fix parsing

* chore: create answer

* chore: create answer
2024-06-08 22:51:19 +08:00
Bartosz Sypytkowski b52369de7f
feat: add workspace settings with disable indexing option (#606)
* feat: add workspace settings with disable indexing option

* chore: added api workspace settings endpoint

* chore: add workspace settings read and write to client api

* chore: add tests for workspace settings client api

* fix: enforce authorization rules on workspace settings api

* chore: fix appflowy indexer test configuration

* chore: add dotenvy to appflowy indexer tests

* chore: override appflowy indexer env vars for tests

* chore: add appflowy indexer preindex env var
2024-06-06 09:26:21 +08:00
Nathan.fooo 6bb2a20888
fix: nginx ai test (#603)
* chore: add config for ai/chat

* chore: update config
2024-06-04 20:11:19 +08:00
Nathan.fooo 50ff40f9d1
chore: add ai completoin scope (#599)
* chore: add ai completoin scope

* chore: fix ci to run test
2024-06-04 10:49:28 +08:00
Nathan.fooo b36715dc24
chore: generate answer manually (#594)
* chore: generate answer manually

* chore: rename

* chore: return reply message id

* chore: save message

* chore: commit schema files
2024-06-03 08:06:23 +08:00
Nathan.fooo 1cc5b58254
chore: update client api (#593) 2024-06-02 20:20:14 +08:00
nathan 33a68983b0 chore: update logs 2024-05-30 21:41:01 +08:00
Nathan.fooo 1537c4d1f6
chore: update chat test (#590) 2024-05-30 17:15:13 +08:00
Mathias Mogensen 06272364b7
feat: get workspace member by user id (#588)
* feat: get workspace member by user id

* chore: remove unneccessary arg
2024-05-30 07:50:58 +08:00
Bartosz Sypytkowski 24429357de
chore: fix after merge compilation issues (#585) 2024-05-29 17:09:49 +08:00
Khor Shu Heng a6bcbd583f
chore: share library for authentication (#576)
* chore: share library for authentication

* fix: return error from requesting app data instead of unwrap

* fix: return internal server error when jwt secret is missing

* chore: remove argon dep from main crate

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-29 16:00:21 +08:00
Bartosz Sypytkowski d3e59fa801
feat: Appflowy-indexer service and /api/search endpoint (#582)
* feat: implement appflowy-indexer and search api endpoint

* chore: store embedding content type

* chore: more descriptive messages for open ai missing key

* chore: fix sqlx errors in tests

* chore: document watcher - change plain text rendering to use depth-first scan

* chore: added nested document blocks example
2024-05-28 21:09:29 +08:00
Nathan.fooo 559d924cd1
feat: stream chat message (#577)
* chore: save author id

* chore: stream response

* chore: stream chat message
2024-05-26 22:44:08 +08:00
Nathan.fooo ae3e075475
chore: Integrate with client-api, update chat message (#574)
* chore: update

* chore: get chat message with next back option

* chore: update next back

* chore: return answer

* chore: return answer

* chore: return answer

* chore: update client api

* chore: fix ci
2024-05-24 15:04:50 +08:00
Nathan.fooo 4eb1e6bceb
chore: generate history using redis steam (#570)
* chore: combine test

* chore: add snapshot test

* chore: add test
2024-05-23 07:57:20 +08:00
Nathan.fooo 05e7f1cda7
Integrate ai chat (#569)
* chore: fix compile

* chore: implement client api

* chore: implement client api

* chore: clippy
2024-05-22 11:18:07 +08:00
Khor Shu Heng 67d9fad7d7
chore: migrate collab storage to appflowy-collaborate (#566)
* chore: migrate collab storage to appflowy-collaborate

* fix: clippy error

* chore: remove handler

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-21 10:06:26 +08:00
Khor Shu Heng fbdcbd6e08
chore: migrate collab actors to appflowy-collaborate (#556) 2024-05-17 12:22:43 +08:00
Nathan.fooo 2fae260e4f
chore: complete text endpoints (#557)
* chore: complete text endpoints
2024-05-16 22:48:44 +08:00
Khor Shu Heng c79267d42a
feat: migrate collab access control to collaboration service (#548)
* feat: migrate collab access control to collaborate service

* feat: migrate workspace access control to separate crate
2024-05-15 15:24:07 +08:00
Nathan.fooo 4c00ddd593
chore: Chat history (#546)
* chore: imple sql curd

* chore: update chat

* chore: select messages

* chore: update test

* chore: update schema

* chore: update
2024-05-11 20:41:21 +08:00
Zack Fu Zi Xiang fd2115f54f
feat: add post method for batch get collab 2024-05-09 11:01:16 +08:00
Zack Fu Zi Xiang cb31a863ac
Merge branch 'main' into fix/batch-get-collab 2024-05-09 10:46:24 +08:00
Nathan.fooo 388d38428b
chore: revamp post realtime endpoint (#536)
* chore: revamp  post realtime endpoint

* chore: fix clippy

* chore: fix fmt
2024-05-08 22:06:37 +08:00
Zack Fu Zi Xiang 5f6e92e80d
Merge branch 'main' into fix/batch-get-collab 2024-05-08 18:02:47 +08:00
Nathan.fooo 70262a1ac4
chore: try to not block actor mailbox (#534)
* chore: add test

* chore: spawn task to handle message

* chore: update error message
2024-05-08 16:54:05 +08:00
Zack Fu Zi Xiang e87e88446c
chore: use http query instead of json for batch get collab 2024-05-08 16:29:27 +08:00
Nathan.fooo c115414f5a
chore: return no content (#523) 2024-05-05 10:52:56 +08:00
Nathan.fooo 79bb510aaf
ci: run history test (#513)
* ci: run history test

* chore: fix ci
2024-05-01 16:28:07 +08:00
Zack Fu Zi Xiang 9adf3f883c
feat: improve email workspace invitations 2024-04-30 11:07:29 +08:00
Zack Fu Zi Xiang 206c7a29ea
chore: merge with main 2024-04-29 23:35:03 +08:00
Zack Fu Zi Xiang 6e74449ab1
feat: accept workspace invite email 2024-04-29 19:40:20 +08:00
Nathan.fooo 7b4b9d0c10
chore: folder data check (#496)
* chore: folder data check

* chore: stop observe remote change when specific type of collaboration object override with incorrect data

* chore: commit sqlx file
2024-04-26 13:20:06 +08:00
Nathan.fooo a0593b5502
chore: refactor return response with object id (#494)
* chore: refactor return response with object id

* chore: fix ci

* chore: fix ci
2024-04-24 12:07:54 +08:00
Nathan.fooo efbe9c4a93
chore: move collab rt crate (#489)
* chore: create appflowy collaborate crate

* chore: remove collab-rt

* chore: fmt

* chore: main
2024-04-22 12:14:41 +08:00
Nathan.fooo 970fbaed57
feat: using transaction to write multiple records (#477)
* chore: impl batch write

* chore: update test

* chore: using save point to optimize write

* chore: update docs

* chore: fix test

* chore: increase write speed

* chore: use redis sorted set

* chore: fallback to write to disk directly when fail to queue

* chore: fix test

* chore: fix test

* chore: update log

* chore: add metrics
2024-04-19 15:22:57 +08:00
Nathan.fooo 4168d72390
chore: using redis conn for each action (#476)
* chore: batch insert

* chore: opti redis conn

* chore: fix test
2024-04-17 11:41:50 +08:00
Nathan.fooo 8ab7815740
chore: reduce pool lock (#475)
* chore: reduce pool lock

* chore: clippy
2024-04-16 20:26:26 +08:00
Nathan.fooo baeeb2c5ce
chore: do not wait insert to redis finish when calling get collab (#468) 2024-04-13 10:26:05 +08:00
Zack 9a5636caec
chore: get collab http (#463)
* chore: get collab http

* fix: use correct version

* fix: collab url

* feat: update collab access control middleware
2024-04-12 11:08:38 +08:00
Nathan.fooo b1c3d779ba
chore: reduce save collab db operations (#455)
* chore: reduce save collab db operations

* chore: clippy

* chore: add test

* chore: add test

* chore: add test
2024-04-08 14:50:51 +08:00
Nathan.fooo b1a6338527
refactor: actix-ws (#453)
* chore: deduplicate ws message

* chore: remove retry when sending client ws message

* chore: prepare client/server actor test
2024-04-07 12:43:29 +08:00
Nathan.fooo 2cc5d75d01
chore: use release build in docker (#440)
* chore: use release build in docker

* chore: remove assert

* chore: update path
2024-04-03 14:30:46 +08:00
Nathan.fooo a18b3d7886
feat: using pending to determine init sync or not (#435)
* chore: optimize sync

* chore: optimize sync

* chore: add test

* chore: update collab commit id

* chore: update

* chore: fix ci
2024-04-01 21:59:41 +08:00
Nathan.fooo d77fb77a3a
feat: validate collab data before writing to disk (#425)
* chore: validate collab

* chore: update test

* chore: partition key

* chore: save collab

* chore: save collab
2024-03-29 14:55:51 +08:00
Nathan.fooo cd9fc34465
refactor: crate folder (#423)
* refactor: crate folder

* chore: update docs

* chore: clippy
2024-03-28 11:06:14 +08:00
Nathan.fooo 66a5033d16
chore: remove component folder (#419) 2024-03-26 14:16:03 +08:00
Nathan.fooo 47649efe18
chore: add connect at timestamp (#410)
* chore: add connect at timestamp

* chore: fix lock issue
2024-03-23 14:00:22 +08:00
Nathan.fooo 6af8ac0f37
chore: remove realtime user trait (#409)
* chore: remove realtime user trait

* chore: fix fmt

* chore: fix clippy
2024-03-23 11:01:50 +08:00
Nathan.fooo c85383b21d
chore: project folder structture (#392)
* chore: docker build

* chore: update

* chore: update

* chore: test stream pubsub

* chore: fix wanrings

* chore: realtime crate

* chore: realtime crate

* chore: realtime crate

* chore: realtime crate

* chore: reanme crates

* chore: reanme crates

* chore: project structure

* chore: disable docker for collab hisotry
2024-03-22 11:24:28 +08:00
Zack Fu Zi Xiang 528dd6ae5b
feat: email redirect to invite page 2024-03-21 14:01:48 +08:00
Zack Fu Zi Xiang 117473d104
feat: leave workspace 2024-03-20 13:07:43 +08:00
Zack Fu Zi Xiang 44ced4036e
Merge branch 'main' into leave-workspace 2024-03-20 09:35:00 +08:00
Zack Fu Zi Xiang c8133302c1
feat: remove storage limit 2024-03-19 22:13:59 +08:00
Zack Fu Zi Xiang 5f220ab0b0
feat: user leave workspace 2024-03-19 21:58:56 +08:00
Nathan.fooo a31c246b67
feat: Access control group (#395)
* chore: access control for group

* chore: access control for workspace

* chore: add test

* chore: enforce group

* chore: update docs
2024-03-19 12:57:08 +08:00
Nathan.fooo e203098517
chore: remove deprecated user logic (#389) 2024-03-17 14:28:37 +08:00
Zack Fu Zi Xiang 1dbffaf4f0
Merge branch 'main' into frontend-admin/user-usage 2024-03-14 17:52:20 +08:00
Zack Fu Zi Xiang 0048533914
feat: total document usage 2024-03-14 17:50:19 +08:00
Zack Fu Zi Xiang 9093363b4f
feat: document storage size 2024-03-14 17:32:30 +08:00
Nathan.fooo 40fc18a499
feat: only enforce when permission change (#382)
* feat: only enforce when permission change

* chore: fix client api ci

* chore: fix test

* chore: fix test
2024-03-13 17:14:43 +08:00
Zack Fu Zi Xiang c293929b11
feat: added tests for inviting member which has not signed up yet 2024-03-06 14:01:57 +08:00
Nathan.fooo 0b0a078b29
ci: fix wasm test (#369)
* ci: fix wasm test

* chore: clippy
2024-03-05 15:03:33 +08:00
Zack Fu Zi Xiang 36c0181032
Merge branch 'main' into workspace-invite-2 2024-03-05 09:34:35 +08:00
Nathan.fooo 0e4442159e
chore: support client version when connect using webscoket (#368)
* chore: support client version when connect using webscoket

* chore: update tokio version

* chore: fix audit

* chore: fix wasm test

* chore: update

* chore: update

* chore: update

* chore: update
2024-03-05 07:22:56 +08:00
Zack Fu Zi Xiang e9ee0d12fa
Merge branch 'main' into workspace-invite 2024-03-04 15:47:57 +08:00
Nathan.fooo 15c03e4f85
refactor: realtime dir (#363)
* refactor: realtime dir

* chore: fmt

* chore: fix test

* chore: fix test

* chore: fix test
2024-03-04 12:53:44 +08:00
Zack Fu Zi Xiang 65d69ef0fd
fix: owner invite permission 2024-03-04 00:59:11 +08:00
Zack Fu Zi Xiang 439a59ed2d
chore: save current progress 2024-03-03 19:06:24 +08:00
Zack Fu Zi Xiang bc3fccfcea
chore: merge with main 2024-03-03 14:37:50 +08:00
Zack Fu Zi Xiang f1e81d59c4
test: crud flow for workspace invites 2024-03-03 14:26:03 +08:00
Nathan.fooo 0e57de98d8
refactor: Access control (#361)
* refactor: access level

* refactor: access control

* refactor: enforce action

* refactor: collab cache

* chore: fix test

* chore: fix test

* chore: fix test

* chore: fix test

* chore: commit migration file

* chore: commit migration file
2024-03-03 12:55:12 +08:00
Zack Fu Zi Xiang e8328fd0a4
feat: workspace invitations 2024-03-03 02:48:01 +08:00
Nathan.fooo 2cf857bd00
feat: optimize websocket message (#351)
* chore: send multiple message

* refactor: message

* refactor: message

* fix: test

* fix: test

* fix: test

* chore: docs

* fix: test

* chore: support compress

* chore: interval retry

* chore: fix test

* chore: merge main

* chore: clippy

* chore: clippy

* ci: fix test

* chore: clippy

* chore: clippy
2024-03-02 02:53:14 +08:00
Zack 79a0dd43c6
feat: Create workspace (#357)
* feat: initialize collab for user after workspace creation

* fix: add permission before adding new collab for new workspace

* chore: simplify logic
2024-02-29 13:02:27 +08:00
Zack Fu Zi Xiang f8b9f623cf
feat: delete all files when workspace is deleted 2024-02-28 18:24:17 +08:00
Nathan.fooo 5ed6a14943
chore: reduce lock required (#353)
* chore: spawn local task

* chore: spawn local task

* chore: spawn local task

* chore: spawn local task

* chore: fix test

* chore: fix test

* chore: clippy

* chore: clippy

* chore: clippy

* chore: clippy
2024-02-27 18:50:44 +08:00
Zack Fu Zi Xiang 7abba9d7ef
feat: add workspace icon field 2024-02-27 11:26:31 +08:00
Nathan.fooo deffd074b9
chore: check collab before insert (#349)
* chore: check collab before insert

* chore: fix audit

* chore: speed test

* chore: add expirate duration for cache encode collab

* chore: fix test
2024-02-25 12:38:07 +08:00
Nathan.fooo 10fc8dd2fe
chore: use Redis cache for enforcer (#348)
* chore: use redis cache

* chore: use redis to cache enforce result

* fix: test

* fix: test

* fix: test

* fix: test
2024-02-24 15:45:53 +08:00
Zack Fu Zi Xiang ba989e64f3
feat: added rename workspace 2024-02-22 11:13:37 +08:00
Zack Fu Zi Xiang ad0f477505
revert to workspace-add-email 2024-02-22 11:09:50 +08:00
Zack Fu Zi Xiang c50f872bb7
Revert "Merge pull request #328 from AppFlowy-IO/workspace-add-email"
This reverts commit 5e7794646a, reversing
changes made to d5869742c2.
2024-02-22 10:48:10 +08:00
Zack Fu Zi Xiang 446518adf6
feat: api: add rename endpoint 2024-02-22 10:31:51 +08:00
Zack Fu Zi Xiang f9d9977076
chore: merge with main 2024-02-20 00:40:42 +08:00
Zack Fu Zi Xiang 32fe1cc24d
feat: send email and create user if adding member but member not exist 2024-02-19 15:40:36 +08:00
Nathan.fooo 815be08d3b
chore: metrics for access control (#326)
* chore: metrics for access control

* chore: fix test
2024-02-19 12:26:35 +08:00
Nathan.fooo 2f0f093331
refactor: access control (#318)
* refactor: dir

* chore: update

* fix: test

* chore: fix test

* chore: clippy

* chore: update config

* chore: update
2024-02-17 01:15:06 +08:00
Nathan.fooo a1494f0d09
feat: Realtime metrics (#309)
* chore: add metric

* chore: add metric

* chore: clippy

* chore: clippy

* ci: set rust version

* ci: set rust version

* ci: set rust version
2024-02-09 12:02:54 +08:00
Nathan.fooo fa13422a05
chore: logs (#302)
* chore: logs

* chore: logs
2024-02-07 13:00:29 +08:00
Nathan.fooo 6f2faff21f
chore: fix log (#301) 2024-02-07 08:15:57 +08:00
Nathan.fooo d851b303bf
refactor: async send ws message in http request (#296)
* refactor: async send ws message in http request

* chore: update

* chore: update logs

* chore: update logs

* chore: clippy

* chore: add device id in header

* chore: fix test

* chore: fix device_id

* chore: fix client id

* chore: ratelimt for client

* chore: bump collab
2024-02-06 15:21:48 +08:00
Nathan.fooo 1a7866e651
chore: use redis to cache collab data (#294)
* chore: use redis to cache collab data

* chore: create table

* chore: use cache

* chore: period check group status

* chore: remove trait

* chore: update logs

* chore: update logs

* chore: update logs

* chore: update logs
2024-02-05 09:54:16 +08:00
Nathan.fooo 9cd1f5ab0a
feat: cache uid (#290)
* feat: cache uid

* chore: remove select uid call

* chore: break loop if mailbox close
2024-02-03 13:08:49 +08:00
Fu Zi Xiang d04eae91e9
feat: add optional name param for workspace creation 2024-01-24 11:58:53 +08:00
Fu Zi Xiang 353065dfbf
feat: add optional name param for workspace creation 2024-01-24 11:35:44 +08:00
Fu Zi Xiang 7c95f6556a
feat: added test case and use triggers 2024-01-23 14:48:25 +08:00
Fu Zi Xiang 27b7b8b5b8
feat: add workspace and delete workspace 2024-01-23 12:10:55 +08:00
Fu Zi Xiang 8ba4559cbd
fix: result return from put blob 2024-01-13 00:53:41 +08:00
Nathan.fooo 79702b1ced
feat: stream create list of collab (#253) 2024-01-11 09:28:28 +08:00
Zack 97d1bb532c
feat: revamp storage api (#251)
* feat: revamp storage api

* feat: client api add method to generate url

* feat: remove some unused deps

* feat: upgrade deps

* feat: add mime to resp
2024-01-11 00:32:11 +08:00
Nathan.fooo 284e2bf638
chore: compress collab data (#249)
* chore: compress collab data

* chore: custom buffer size

* chore: compression ws message

* chore: update log

* chore: update log

* chore: update client max body size

* chore: update client api

* docs: update docs

* chore: retry transaction

* chore: update timeout

* chore: stop previous loop

* chore: log ping error

* chore: log send error

* chore: log error

* chore: update log

* chore: async compress

* feat: stream payload

* chore: update load

* chore: update

* chore: update log

* chore: fix cmt

* fix: client message from http

* chore: bump collab
2024-01-10 08:05:57 +08:00
Nathan.fooo 4886d3d474
chore: rename error message (#247) 2024-01-05 07:29:06 +08:00
Nathan.fooo 8e38ae7b69
refactor: remove rust-s3, sqlx from client-api (#245)
* refactor: remove rust-s3, sqlx from client-api

* chore: fmt

* chore: add ci check

* chore: add ci check

* chore: add ci check
2024-01-05 06:12:01 +08:00
Nathan.fooo c5fe447250
feat: bump collab (#243)
* chore: bump collab

* chore: bump collab

* chore: bump collab
2024-01-03 08:40:26 +08:00
Nathan.fooo 6d0612ffaf
feat: encode batch create collab data as binary (#242) 2024-01-03 06:07:21 +08:00
Nathan.fooo a455c9de8e
chore: refactor logs (#236)
* chore: add logs

* chore: return ws state

* fix: init the collab from disk
2023-12-27 05:20:40 +08:00
Nathan.fooo a07cc0b903
feat: Insert multiple collab (#233)
* chore: remove panic hook

* chore: support insert multiple collabs
2023-12-25 02:03:41 +08:00
Nathan.fooo 69ed6ff4e8
feat: create snapshot for collab (#221) 2023-12-17 13:38:55 +08:00
Nathan.fooo cd2c0a3114
fix: casbin access control (#214)
* chore: fix access control

* fix: create user update collab permissions

* chore: select workspace role it's not found in cache

* fix: encode buffer error

---------

Co-authored-by: Jiraffe7 <twajxjiraffe@gmail.com>
2023-12-16 08:27:57 +08:00
Jiraffe7 36ef0f13b0
feat: casbin access control (#178)
* feat: casbin for access control

* fix: method to generate database url instead of ToString

* fix: hold write lock when modifying policies

* chore: fix compile

* fix: remove db constraint for lib integration tests

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-12-11 11:08:15 +08:00
Nathan.fooo c0f5c2ce7e
feat: fix flaky test (#202)
* test: fix

* fix: flaky test
2023-12-09 12:48:46 +08:00
Nathan.fooo ee31a680b5
chore: unlimited json payload size (#197)
* chore: return ok when token is empty

* chore: remove json payload size limit

* chore: clippy
2023-12-06 14:21:11 +08:00
Nathan.fooo 79a02edd00
feat: notify user via websocket after profile changed (#164)
* feat: notify user via websocket after profile changed

* chore: update

* chore: update
2023-11-14 12:47:10 +08:00
Nathan.fooo 54ef875f5f
feat: notify user profile change (#163)
* feat: send user profile changes via ws

* test: add tests
2023-11-13 22:58:18 +08:00
Nathan.fooo 04989e9485
feat: Upload collab limit (#160)
* chore: add logs

* feat: update upload collab limit
2023-11-12 16:03:48 +08:00
Nathan.fooo f626e4a3b2
chore: log for broadcast (#154)
* chore: return not found

* chore: add logs

* chore: handle ws error

* chore: add logs

* chore: add logs

* chore: ping

* chore: ping

* chore: add log

* chore: update

* chore: update

* chore: update
2023-11-09 14:24:32 +08:00
Nathan.fooo f1dad07caa
feat: http init sync (#151)
* fix: bugs

* chore: support message type

* refactor: ws message

* feat: post realtime message

* feat: use proto to reduce message size

* ci: update

* ci: install protoc

* ci: update docker

* chore: update ci

* ci: update docker

* ci: fix

* ci: fix

* ci: fix
2023-11-08 15:41:39 +08:00
Nathan.fooo 41e01b91fc
feat: update collab rev and fix some bugs (#148)
* chore: remove todo

* chore: update collab rev

* chore: fmt

* chore: fmt

* chore: cache collab user

* fix: internal error

* ci: fix test

* chore: update collab rev

* ci: fix
2023-11-05 22:43:20 +08:00
Nathan.fooo d088fd2ec8
feat: ws frame size (#146)
* chore: update ws frame size

* chore: update

* chore: update frame size to 64 kb

* chore: fix merge

* chore: update

* chore: update collab rev

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update ws state

* chore: update pg max connect

* chore: fix crash
2023-11-04 21:42:06 +08:00
Zack 37a759ead6
feat: Request Id (#144)
* feat: added basic monitoring stack

* docs(grafana): import steps and recommended imports

* feat(metric): api metrics

* feat(prometheus): add appflowy cloud metric scraping job

* fix: put back tracing logger

* feat: middleware for path counter

* feat: added gotrue to prometheus metrics

* feat: added nginx to prometheus metrics

* feat: added latency and result to metrics

* feat(nginx): set request id to header

* feat: added request id in nginx

* feat: simplify metric collection

* feat: added simple grafana dashboard for appflowy cloud

* feat(logging): added logging stack

* docs: improve monitoring docs

* docs: added known common issues

* feat: give persistence

* feat: add exemplars

* fix: docker compose volumes

* feat(grafana): add trace id to prometheus datasource

* feat(grafana): improve dashboard to include exemplars

* feat: request id middleware

* docs: move doc to more relevant place

* chore: clean up logs

* feat: try json fmt

* feat: remove pretty

* feat: logging use json in prod but pretty in local
2023-11-03 16:57:05 +08:00
Nathan.fooo 937e3bd9b2
fix: fix some bugs (#145)
* chore: update

* chore: pub error code

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update max frame size

* chore: update max frame size

* chore: ws buffer size

* chore: update
2023-11-03 14:15:08 +08:00
Nathan.fooo 9c911a3a94
refactor: Error (#143)
* chore: remove api

* chore: update log

* refactor: error

* chore: fmt

* ci: fix test

* ci: fix

* chore: remove unstable
2023-11-02 17:17:34 +08:00
Zack 1cc1158d36
feat: Metrics (#141)
* feat: added basic monitoring stack

* docs(grafana): import steps and recommended imports

* feat(metric): api metrics

* feat(prometheus): add appflowy cloud metric scraping job

* fix: put back tracing logger

* feat: middleware for path counter

* feat: added gotrue to prometheus metrics

* feat: added nginx to prometheus metrics

* feat: added latency and result to metrics

* feat(nginx): set request id to header

* feat: added request id in nginx

* feat: simplify metric collection

* feat: added simple grafana dashboard for appflowy cloud

* feat(logging): added logging stack

* docs: improve monitoring docs

* docs: added known common issues

* feat: give persistence

* feat: add exemplars

* fix: docker compose volumes

* feat(grafana): add trace id to prometheus datasource

* feat(grafana): improve dashboard to include exemplars
2023-11-02 15:34:00 +08:00
Nathan.fooo 5020d4738a
feat: Integrate workspace member (#140)
* chore: update client api

* chore: return user name in the workspace memeber

* chore: optimize logs
2023-10-25 20:44:30 +08:00
Nathan.fooo 8c1c5d0cd9
feat: insert uid instead of gen it from db (#139) 2023-10-25 12:18:33 +08:00
Nathan.fooo 49f994488a
chore: redirect url (#134)
* chore: redirect url

* chore: stop ws conn if error is auth error

* chore: add query params

* chore: fix clippy
2023-10-24 00:40:44 +08:00
Nathan.fooo 7c503372e0
feat: client-api integration: save user meta data (#133)
* chore: update

* feat: get user workspace info

* feat: return list of workspace

* feat: return latest workspace id

* feat: latest workspace id

* test: add tests
2023-10-23 15:03:31 +08:00
Nathan.fooo d0d2e916a7
refactor: hiding sqlx row entitis from client api (#132) 2023-10-22 22:10:03 +08:00
nathan ec13239bdd chore: update dto 2023-10-22 21:10:30 +08:00
Nathan.fooo 7a309c6f69
feat: insert user metadata (#131) 2023-10-22 19:21:23 +08:00
Nathan.fooo 1aba1f0cf4
feat: workspace usage test (#129)
* feat: workspace usage test

* test: add tests
2023-10-22 17:52:10 +08:00
Nathan.fooo fe5ef43333
refactor: workspace member crud (#128)
* refactor: workspace member crud

* refactor: update user api

* test: fix test
2023-10-21 20:50:32 +08:00
Nathan.fooo 417d9f1d41
feat: cache workspace member & fix some bugs (#127)
* chore: insert collab member when inserting workspace member

* refactor: test directory

* chore: remove triggers

* test: add more test

* chore: cache workspace memeber role

* chore: update test
2023-10-20 19:43:36 +08:00
Nathan.fooo cb9cdb9280
feat: collab permission test (#126)
* chore: check permission for collab storage proxy

* feat: add permission tests

* chore: skip log
2023-10-19 17:12:21 +08:00
Nathan.fooo 9dc7bbeee0
feat: Collab access control (#120)
* chore: check collab message with access permission

* chore: imple collab permission service

* refactor: migrations

* chore: collab member ops

* chore: collab permission

* chore: update can edit workspace collab

* chore: fix test

* feat: fetch collab members

* chore: fix test

* chore: fix client api

* chore: check permission for collab storage proxy
2023-10-17 14:00:04 +08:00
Nathan.fooo 88be0c2433
refactor: access control (#118)
* refactor: access control

* chore: fix test
2023-10-15 10:56:56 +08:00
Nathan.fooo 3e73adc82d
feat: workspace manager & test (#117)
* chore: add workspace tests

* chore: add slqx files

* feat: update workspace member role

* chore: update
2023-10-14 12:23:23 +08:00
Nathan.fooo 5c58f95f9f
feat: workspace access control (#115)
* feat: workspace access control

* feat: extact workspace id from path

* feat: create member with permission set

* chore: update sqlx files

* chore: fix cargo toml warn

* chore: clippy
2023-10-13 20:02:29 +08:00
nathan 7b77805d8d chore: add logs 2023-10-13 10:37:28 +08:00
Nathan.fooo 3611c2076e
feat: update ws connect setting (#111)
* chore: update ws connect setting

* chore: remove token when token is invalid

* chore: close the ws connect when overflow happened

* fix: reset timing

* fix: reset timing

* chore: sleep if timeout not reach

* chore: sleep if timeout not reach
2023-10-12 16:51:44 +08:00
Nathan.fooo ef726092be
refactor: file storage (#110)
* refactor: return file id after save to disk

* test: add tests

* chore: mine type

* test: disbale two client sync test

* chore: get file with url

* refactor: client api
2023-10-11 21:36:57 +08:00
Nathan.fooo 089b3046ab
chore: implement batch get (#106)
* chore: implement batch get

* chore: add request id and update the local_server.sh

* chore: update collab commit id
2023-10-08 23:53:16 +08:00
Nathan.fooo d3186cc07a
feat: websocket config (#99)
* chore: enable tls feature

* chore: update ws client

* chore: update ws client

* chore: expost database entities

* chore: update ws nginx config

* chore: rename error file

* chore: fix clippy
2023-10-05 17:43:50 +08:00
Zack 583f82b7fb
refactor: collab http (#89)
* refactor: collab http

* fix: sqlx uuid in types client_api
2023-10-04 16:59:03 +08:00
Nathan.fooo 19ee96db4a
chore: rename crate (#82) 2023-09-28 22:20:48 +08:00
Zack a6eb8607c9
feat: async read for file upload (#80)
* feat: async read for file upload

* feat: async read for file download

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-28 14:53:41 +08:00
Nathan.fooo b8bd53ff5c
feat: create collab sync crate (#79)
* feat: create collab sync crate

* test: add tst

* test: add test

* test: update tests

* chore: clippy

* chore: clippy

* chore: clippy

* chore: add retry when subscribe group

* chore: disable realtime test
2023-09-28 09:28:46 +08:00
Zack ff6a8e1eaf
feat: File api (#70)
* feat: s3 minio wip

* feat: s3 minio bucket create idempotent

* feat: put storage setting into configurations

* chore: clippy lint

* feat: add setting to base config

* feat: add configuration for prod

* fix: allow use different minio host

* feat: add server file storage

* fix: add missing file

* feat: add code template

* feat: add http api

* feat: added file upload to client api

* feat: database metadata impl

* fix: added missing files

* feat: added test cases and improve impl

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-27 10:21:37 +08:00
Nathan.fooo b729e3529d
chore: bump collab version (#78)
* chore: bump collab version

* chore: bump collab version

* chore: update

* chore: update

* chore: update

* chore: update tests

* chore: clippy

* chore: clippy

* chore: clippy
2023-09-26 22:52:41 +08:00
Nathan.fooo 0883ae94b8
test: sign in url test (#75)
* test: sign in url test

* fix: Tungstenite allows remote attackers to cause a denial of service

* chore: enable feature

* chore: update
2023-09-24 13:03:24 +08:00
Zack f1a16058ce
feat: Gotrue admin api (#73)
* feat: set up admin account and auto confirm during appflowy start

* feat: client auth against gotrue whenever possible

* feat: admin add user

* feat: implement admin add user

* feat: generate registered user

* fix: enable cloud feature for client_api

* fix: test same user fix
2023-09-23 17:07:02 +08:00
Nathan.fooo b0622efe08
Feat update user (#69)
* chore: update user

* refactor: token

* chore: update
2023-09-22 20:28:39 +08:00
Zack df9c71edb2
feat: returns if user is new for both oauth and password login (#67)
* feat: returns if user is new for both oauth and password login

* test: add test for firt time sign_in and subsequent

* chore: remove uneeded use

* fix: compile

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-21 14:05:25 +08:00
Zack bbc913e45f
feat: support name for oauth sign in (#65)
* feat: support name for oauth sign in

* fix: add sqlx metadata

* feat: improve api, add name modifications

* fix: add sqlx query metadata
2023-09-21 09:28:07 +08:00
Zack 0676db7ed6
feat: Workspace members (#63)
* fix: create user if they sign up with oauth

* feat: add workspaces_members add and remove operations

* feat: add insert workspace members

* feat: attach db ops to api ops

* refactor: separate user and workspace scope

* chore: add sqx query metadata

* chore: add 2nd registered user for testing

* ci: use valid email

* ci: fix registered password invalid

* fix: use email instead of uid

* fix: use email instead of uid sqlx query metadata

* feat: add test for workspace member

* test: added workspace add and delete with checking

* ci: fix email
2023-09-19 20:59:16 +08:00
Nathan.fooo 7ae645a7c1
test: add ws reconnect test (#58)
* test: add test

* test: add reconnect
2023-09-18 11:42:32 +08:00
Nathan.fooo 939ea29c3b
feat: Realtime multiple conn test(#45)
* chore: multiple connection with same user

* chore: connect with device id

* fix: multiple connection

* test: add more test

* chore: fmt
2023-09-15 09:59:37 +08:00
Zack b3be09e264
feat: added refresh ability on server and client (#41)
* feat: added refresh ability on server and client

* fix: use refresh token for refresh and add test case

* chore: cargo fmt --all

* chore: cargo clippy

* fix: cargo clippy

* test: added async mutex for registered user for consistency

* fix: remove unneeded files

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-14 15:58:18 +08:00
Nathan.fooo e03a6ce587
test: fix collab edit test (#42)
* test: fix

* chore: cargo sqlx check workspce

* chore: remove rocksdb deps
2023-09-14 14:05:15 +08:00
Nathan.fooo b1adcc7919
test: websocket connect and then edit collab test (#40)
* refactor: realtime dir

* test: add test

* test: fix
2023-09-14 09:34:24 +08:00
Zack 8d11d7b6cf
feat: support external oauth (#35)
* feat: support external oauth

* chore: cargo fmt

* doc: add docs for google oauth2

* fix: use btree map and get for oauth

* feat: client to open url for oauth login

* feat: add test case for oauth not available

* feat: add placeholder for sign_in_url

* fix: use

* feat: client sign up with url

* feat: add sign_in_with_url simple test

* fix: use url fragments and string split

* fix: test case and gotrue error handling

* fix: return error and remove unwrap

* chore: remove unused file not in module tree

* fix: test cases

* chore: add cfg feature cloud to isolate dependencies
2023-09-13 21:08:13 +08:00
Nathan.fooo c42158b7cb
test: add ws connect test (#39)
* test: add ws connect test

* chore: format log
2023-09-13 17:04:42 +08:00
Nathan.fooo 107627f4d8
feat: implement snapshot handler (#38) 2023-09-13 15:25:51 +08:00
Nathan.fooo 3aa4d20ad3
chore: reduce deps of client_api crate (#37) 2023-09-13 13:12:07 +08:00
Nathan.fooo b32fcff4d4
test: Collab handler test (#36)
* chore: implement collab requests

* test: add more tests

* chore: update scope path
2023-09-13 12:52:22 +08:00
Nathan.fooo f0f58f98f3
feat: create collab handlers (#34)
* feat: logged user after sign up/sign in

* test: fix test

* chore: update sqlx files

* chore: add router

* chore: parser user uuid from token
2023-09-12 21:52:25 +08:00
Zack 0d59211e55
feat: expose workspace and profile api (#32)
* feat: expose workspace and profile api

* feat: add impl for client_api

* feat: add test case for workspace and profile
2023-09-12 16:14:14 +08:00
Nathan.fooo 9ac53dca8e
chore: implement collab storage using postrges (#31)
* chore: implement collab storage using postrges

* chore: update collab rev

* chore: update .sqlx files
2023-09-12 15:36:33 +08:00
Zack 853f089ca0
feat: test gotrue when app start, use state for gotrue client, use go… (#24)
* feat: test gotrue when app start, use state for gotrue client, use gotrue UUID instead of BIGINT for user uid

* feat: backward compatibility with native authetication

* fix: native auth use uuid instead

* fix: sqlx preparation

* fix: sqlx metadata and unused variable

* fix: clippy

* fix: create workspace if not exist after sign in

* chore: update .sqlx with new query

* feat: query workspace

* revert: revert using i64 for user id

* fix: update sqlx queries

* fix: update .sqlx metadata

* feat: only use uuid from sqlx types, select user_profile view

* ci: add version check

* ci: add missing sqlx files

* chore: cargo fmt
2023-09-11 17:16:41 +08:00
Nathan.fooo 3cd3240bfc
chore: move realtime test (#26)
* chore: fix tests

* chore: add test server

* chore: move tests
2023-09-10 19:35:29 +08:00
Nathan.fooo 4a56cbb91f
refactor: refactor app data struct (#23)
* refactor: refactor app data struct to reduce memory cost

* refactor: rename AppData to AppResponse

* chore: implement traits

* chore: remove default features of actix-web
2023-09-10 11:56:22 +08:00
Zack d4577f5e61
Feat: enrich http dto (#20)
* feat: add scripts for easier dev

* feat: add Json<T> as resp type for http server

* feat: restructure server and client shared entity

* feat: restructure server and client shared entity

* feat: use serde_repr to deserialize enum into json number

* feat: minor adjustment to results and server error

* feat: test: fix unconfirmed email message

* feat: test: add test case for existing user

* feat: test: modified api for sign_out

* feat: test: add test case pass
2023-09-08 20:01:28 +08:00
Nathan.fooo c1f8d79169
chore: separate crates (#18) 2023-09-07 19:40:11 +08:00
Nathan.fooo f9825d4221
feat: refactor server collab (#15)
* refactor: collab storage

* chore: fix test

* chore: update sqlx version

* chore: disable realtime test

* chore: sqlx prepare

* chore: remove init.sql
2023-09-06 18:22:40 +08:00
Zack c8ae099e90
feat: Auth server integration (#9)
* feat: integrate signup with gotrue auth server

* feat: added email and password sign in via auth server

* feat: added sign in scenarios and client handling

* fix: added missing files

* feat: refine api and store access token in client

* feat: impl token decode and placeholder for signout

* feat: added impl for logout with auth server

* feat: auth server dependency injection, client sign_out impl

* feat: abstract verify token, add placeholder for update endpoint

* feat: make authorization as a param of http handler request

* feat: add update_user to client of auth server

* feat: add client code and test cases for password change

* chore: rename appflowy-server to appflowy-cloud

* ci: upgrade reqwest version to fix audit issue

* ci: fix again

* ci: added gotrue env secrets

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-04 11:23:40 +08:00
Nathan.fooo 8b9e6584d2
feat: ws reconnect (#5)
* feat: ws reconnect

* chore: update collab rev
2023-05-10 20:54:10 +08:00
Nathan.fooo 90ae1d5fb6
Feat/ws test (#4)
* test: ws test

* test: update

* test: update

* test: sync update

* feat: ws test
2023-05-10 20:26:30 +08:00