Commit Graph

189 Commits

Author SHA1 Message Date
Bartosz Sypytkowski 3721d209f2 chore: normalize URL paths in appflowy collab server 2024-06-12 06:21:56 +02:00
Nathan.fooo 56b5c4c843
chore: disable old version history test (#604) 2024-06-04 20:41:31 +08:00
Nathan.fooo 50ff40f9d1
chore: add ai completoin scope (#599)
* chore: add ai completoin scope

* chore: fix ci to run test
2024-06-04 10:49:28 +08:00
Khor Shu Heng a6bcbd583f
chore: share library for authentication (#576)
* chore: share library for authentication

* fix: return error from requesting app data instead of unwrap

* fix: return internal server error when jwt secret is missing

* chore: remove argon dep from main crate

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-29 16:00:21 +08:00
Bartosz Sypytkowski d3e59fa801
feat: Appflowy-indexer service and /api/search endpoint (#582)
* feat: implement appflowy-indexer and search api endpoint

* chore: store embedding content type

* chore: more descriptive messages for open ai missing key

* chore: fix sqlx errors in tests

* chore: document watcher - change plain text rendering to use depth-first scan

* chore: added nested document blocks example
2024-05-28 21:09:29 +08:00
Nathan.fooo ae3e075475
chore: Integrate with client-api, update chat message (#574)
* chore: update

* chore: get chat message with next back option

* chore: update next back

* chore: return answer

* chore: return answer

* chore: return answer

* chore: update client api

* chore: fix ci
2024-05-24 15:04:50 +08:00
Nathan.fooo 4eb1e6bceb
chore: generate history using redis steam (#570)
* chore: combine test

* chore: add snapshot test

* chore: add test
2024-05-23 07:57:20 +08:00
Nathan.fooo 05e7f1cda7
Integrate ai chat (#569)
* chore: fix compile

* chore: implement client api

* chore: implement client api

* chore: clippy
2024-05-22 11:18:07 +08:00
Khor Shu Heng 67d9fad7d7
chore: migrate collab storage to appflowy-collaborate (#566)
* chore: migrate collab storage to appflowy-collaborate

* fix: clippy error

* chore: remove handler

---------

Co-authored-by: nathan <nathan@appflowy.io>
2024-05-21 10:06:26 +08:00
Zack ec7eb54bfc
chore: Self host improvement (#561)
* fix: remove multiple ways to specify database name

* feat: resent email for already invited user

* feat: mailer address from smtp username

* feat: allow user defined smtp port
2024-05-18 08:33:08 +08:00
Nathan.fooo 2736fa60a7
chore: integrate redis stream (#559)
* chore: integrate redis stream

* chore: make CollabUpdateStreaming::send_update synchronous

---------

Co-authored-by: Bartosz Sypytkowski <b.sypytkowski@gmail.com>
2024-05-18 08:28:57 +08:00
Khor Shu Heng fbdcbd6e08
chore: migrate collab actors to appflowy-collaborate (#556) 2024-05-17 12:22:43 +08:00
Nathan.fooo 2fae260e4f
chore: complete text endpoints (#557)
* chore: complete text endpoints
2024-05-16 22:48:44 +08:00
Khor Shu Heng c79267d42a
feat: migrate collab access control to collaboration service (#548)
* feat: migrate collab access control to collaborate service

* feat: migrate workspace access control to separate crate
2024-05-15 15:24:07 +08:00
Nathan.fooo 4c00ddd593
chore: Chat history (#546)
* chore: imple sql curd

* chore: update chat

* chore: select messages

* chore: update test

* chore: update schema

* chore: update
2024-05-11 20:41:21 +08:00
Khor Shu Heng d9d874ab8a
feat: migrate collab cache to collaborate service (#524)
Co-authored-by: nathan <nathan@appflowy.io>
2024-05-09 23:12:13 +08:00
Nathan.fooo 388d38428b
chore: revamp post realtime endpoint (#536)
* chore: revamp  post realtime endpoint

* chore: fix clippy

* chore: fix fmt
2024-05-08 22:06:37 +08:00
Nathan.fooo 70262a1ac4
chore: try to not block actor mailbox (#534)
* chore: add test

* chore: spawn task to handle message

* chore: update error message
2024-05-08 16:54:05 +08:00
Nathan.fooo 79bb510aaf
ci: run history test (#513)
* ci: run history test

* chore: fix ci
2024-05-01 16:28:07 +08:00
Nathan.fooo af75e8a2c0
chore: update configuration for ai docker (#512)
* chore: update configuration for ai docker

* chore: add script to test ci docker

* chore: update docs

* chore: update docs
2024-04-30 19:37:52 +08:00
Zack Fu Zi Xiang 8f2182900c
chore: merge with main 2024-04-29 23:39:43 +08:00
Zack Fu Zi Xiang 206c7a29ea
chore: merge with main 2024-04-29 23:35:03 +08:00
Nathan.fooo 5bebc6a2b2
chore: Integrate appflowy ai (#504)
* chore: remove client api

* chore: add summarize row test

* chore: merge main

* chore: fix ci

* chore: update docker compose file
2024-04-29 22:23:40 +08:00
Zack Fu Zi Xiang 9a25b92e57
fix: ci and use static file for template 2024-04-29 21:25:32 +08:00
Zack Fu Zi Xiang 6e74449ab1
feat: accept workspace invite email 2024-04-29 19:40:20 +08:00
Khor Shu Heng cbedbcb748
chore: patch for RUSTSEC-2024-0336 (#503) 2024-04-28 13:58:28 +08:00
Nathan.fooo efbe9c4a93
chore: move collab rt crate (#489)
* chore: create appflowy collaborate crate

* chore: remove collab-rt

* chore: fmt

* chore: main
2024-04-22 12:14:41 +08:00
Nathan.fooo 970fbaed57
feat: using transaction to write multiple records (#477)
* chore: impl batch write

* chore: update test

* chore: using save point to optimize write

* chore: update docs

* chore: fix test

* chore: increase write speed

* chore: use redis sorted set

* chore: fallback to write to disk directly when fail to queue

* chore: fix test

* chore: fix test

* chore: update log

* chore: add metrics
2024-04-19 15:22:57 +08:00
Nathan.fooo 4168d72390
chore: using redis conn for each action (#476)
* chore: batch insert

* chore: opti redis conn

* chore: fix test
2024-04-17 11:41:50 +08:00
Nathan.fooo f3279e9b4e
feat: implement grpc for history server (#470)
* ci: docker compose for ci

* chore: add test

* chore: imple grpc

* chore: fix test

* chore: fix ci
2024-04-16 14:10:56 +08:00
Nathan.fooo 4d019728ba
refactor: insert encode collab object in redis (#469)
* chore: insert collab redis cache with timestamp

* chore: add test

* chore: update expire time

* chore: update

* chore: fix test
2024-04-13 22:31:36 +08:00
Nathan.fooo b1c3d779ba
chore: reduce save collab db operations (#455)
* chore: reduce save collab db operations

* chore: clippy

* chore: add test

* chore: add test

* chore: add test
2024-04-08 14:50:51 +08:00
Nathan.fooo b1a6338527
refactor: actix-ws (#453)
* chore: deduplicate ws message

* chore: remove retry when sending client ws message

* chore: prepare client/server actor test
2024-04-07 12:43:29 +08:00
Nathan.fooo 40f88a6231
fix: group might not be ready to receive message (#445) 2024-04-04 15:13:55 +08:00
Nathan.fooo 2cc5d75d01
chore: use release build in docker (#440)
* chore: use release build in docker

* chore: remove assert

* chore: update path
2024-04-03 14:30:46 +08:00
Nathan.fooo 6c96d05e2f
feat: introduce custom function in casbin model that supports compari… (#433)
* feat: introduce custom function in casbin model that supports comparing role or level

* chore: update

* chore: log

* chore: log

* chore: remove cache
2024-03-31 19:54:41 +08:00
Khor Shu Heng ccd11f0b02
refactor: make access control a separate package (#422)
Co-authored-by: nathan <nathan@appflowy.io>
2024-03-30 11:06:49 +08:00
Nathan.fooo 6af8ac0f37
chore: remove realtime user trait (#409)
* chore: remove realtime user trait

* chore: fix fmt

* chore: fix clippy
2024-03-23 11:01:50 +08:00
Nathan.fooo c85383b21d
chore: project folder structture (#392)
* chore: docker build

* chore: update

* chore: update

* chore: test stream pubsub

* chore: fix wanrings

* chore: realtime crate

* chore: realtime crate

* chore: realtime crate

* chore: realtime crate

* chore: reanme crates

* chore: reanme crates

* chore: project structure

* chore: disable docker for collab hisotry
2024-03-22 11:24:28 +08:00
Nathan.fooo d1c82b7811
chore: enable access control by env (#394) 2024-03-18 19:34:44 +08:00
Nathan.fooo e203098517
chore: remove deprecated user logic (#389) 2024-03-17 14:28:37 +08:00
Nathan.fooo 08693f2d72
chore: integrate appflowy ai (#388)
* chore: integrate appflowy ai

* chore: disable ai test
2024-03-16 21:43:02 +08:00
Nathan.fooo 40fc18a499
feat: only enforce when permission change (#382)
* feat: only enforce when permission change

* chore: fix client api ci

* chore: fix test

* chore: fix test
2024-03-13 17:14:43 +08:00
Zack Fu Zi Xiang cccd5d5969
fix: hide admin password 2024-03-07 10:08:12 +08:00
Zack Fu Zi Xiang c293929b11
feat: added tests for inviting member which has not signed up yet 2024-03-06 14:01:57 +08:00
Zack Fu Zi Xiang 9b28edb5bc
feat: use email for af workspace invitation table 2024-03-06 11:29:07 +08:00
Nathan.fooo 15c03e4f85
refactor: realtime dir (#363)
* refactor: realtime dir

* chore: fmt

* chore: fix test

* chore: fix test

* chore: fix test
2024-03-04 12:53:44 +08:00
Nathan.fooo 0e57de98d8
refactor: Access control (#361)
* refactor: access level

* refactor: access control

* refactor: enforce action

* refactor: collab cache

* chore: fix test

* chore: fix test

* chore: fix test

* chore: fix test

* chore: commit migration file

* chore: commit migration file
2024-03-03 12:55:12 +08:00
Nathan.fooo 5ed6a14943
chore: reduce lock required (#353)
* chore: spawn local task

* chore: spawn local task

* chore: spawn local task

* chore: spawn local task

* chore: fix test

* chore: fix test

* chore: clippy

* chore: clippy

* chore: clippy

* chore: clippy
2024-02-27 18:50:44 +08:00
Nathan.fooo 10fc8dd2fe
chore: use Redis cache for enforcer (#348)
* chore: use redis cache

* chore: use redis to cache enforce result

* fix: test

* fix: test

* fix: test

* fix: test
2024-02-24 15:45:53 +08:00
Nathan.fooo b71e443d3f
refactor: write snapshot (#345)
* chore: increase max connection size of pg (#344)

* chore: refactor snapshot

* chore: fix test

* chore: fix test

* chore: fix test

* chore: metric

* chore: clippy

* chore: fix test

* chore: fix test
2024-02-23 14:36:06 +08:00
nathan f196208c9b chore: print feature 2024-02-23 05:40:35 +08:00
Zack Fu Zi Xiang c50f872bb7
Revert "Merge pull request #328 from AppFlowy-IO/workspace-add-email"
This reverts commit 5e7794646a, reversing
changes made to d5869742c2.
2024-02-22 10:48:10 +08:00
Zack Fu Zi Xiang 4413f61f26
Merge branch 'main' into workspace-add-email 2024-02-20 10:16:58 +08:00
Nathan.fooo 8e36792dd9
fix: metrics (#331) 2024-02-20 08:34:57 +08:00
Nathan.fooo 5cd16d7544
chore: load user id and uuid when start (#329)
* chore: load user id and uuid when start

* chore: fix test

* chore: fix test
2024-02-20 05:43:33 +08:00
Zack Fu Zi Xiang f9d9977076
chore: merge with main 2024-02-20 00:40:42 +08:00
Zack Fu Zi Xiang 32fe1cc24d
feat: send email and create user if adding member but member not exist 2024-02-19 15:40:36 +08:00
Nathan.fooo 815be08d3b
chore: metrics for access control (#326)
* chore: metrics for access control

* chore: fix test
2024-02-19 12:26:35 +08:00
Nathan.fooo f713ebedaa
chore: cache policy when load (#325) 2024-02-19 10:01:40 +08:00
Nathan.fooo c9a3cd6b1b
chore: rename dir (#324)
* chore: rename dir

* chore: clippy
2024-02-19 08:28:32 +08:00
Nathan.fooo 44d2a6809f
chore: fix migration miss (#321) 2024-02-18 17:38:14 +08:00
Nathan.fooo 2f0f093331
refactor: access control (#318)
* refactor: dir

* chore: update

* fix: test

* chore: fix test

* chore: clippy

* chore: update config

* chore: update
2024-02-17 01:15:06 +08:00
Zack e1ac591ea0
Merge pull request #310 from AppFlowy-IO/admin-frontend/self-host-env
Admin frontend/self host env
2024-02-09 15:15:56 +08:00
Nathan.fooo a1494f0d09
feat: Realtime metrics (#309)
* chore: add metric

* chore: add metric

* chore: clippy

* chore: clippy

* ci: set rust version

* ci: set rust version

* ci: set rust version
2024-02-09 12:02:54 +08:00
Zack Fu Zi Xiang 8626b54de8
fix: redirect users after signup via magiclink 2024-02-09 02:13:11 +08:00
Nathan.fooo d497f2ab62
fix: pg lock (#299)
* chore: fix pg lock

* chore: fix pg lock
2024-02-06 13:44:39 +08:00
Zack Fu Zi Xiang f787c32a62
feat: added pg connection lifetime 2024-02-06 08:17:22 +08:00
Nathan.fooo 1a7866e651
chore: use redis to cache collab data (#294)
* chore: use redis to cache collab data

* chore: create table

* chore: use cache

* chore: period check group status

* chore: remove trait

* chore: update logs

* chore: update logs

* chore: update logs

* chore: update logs
2024-02-05 09:54:16 +08:00
Nathan.fooo 9cd1f5ab0a
feat: cache uid (#290)
* feat: cache uid

* chore: remove select uid call

* chore: break loop if mailbox close
2024-02-03 13:08:49 +08:00
Nathan.fooo e1b35a1a24
feat: enable cors in nginx (#276)
* feat: enable cors in nginx

* chore: update

* chore: fix ci
2024-01-29 13:14:50 +08:00
Fynn Petersen-Frey a39395d90c S3 compatiblity with garage 2023-12-28 15:54:23 +01:00
Fu Zi Xiang 04e00dea95
feat: add more initialization logs 2023-12-20 20:38:11 -08:00
Zack a7b259ad20
feat: use only env var for appflowy cloud (#224)
* feat: use only env var for appflowy cloud

* fix: jwt local testing

* fix: security audit

* feat: update docker deploy configs

* fix: test utils dotenvy

* fix: test try sqlx offline

* fix: add gotrue configs for appflowy

* fix: redis uri in docker
2023-12-19 08:45:42 +08:00
JesseBot 36c792915c feat: add additional info logging to init_state function in application.rs 2023-12-12 12:16:38 +01:00
Jiraffe7 36ef0f13b0
feat: casbin access control (#178)
* feat: casbin for access control

* fix: method to generate database url instead of ToString

* fix: hold write lock when modifying policies

* chore: fix compile

* fix: remove db constraint for lib integration tests

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-12-11 11:08:15 +08:00
Nathan.fooo c0f5c2ce7e
feat: fix flaky test (#202)
* test: fix

* fix: flaky test
2023-12-09 12:48:46 +08:00
Nathan.fooo ee31a680b5
chore: unlimited json payload size (#197)
* chore: return ok when token is empty

* chore: remove json payload size limit

* chore: clippy
2023-12-06 14:21:11 +08:00
Nathan.fooo 0e2ec381df
chore: implement symmetric encrypt (#192) 2023-12-04 11:50:56 +08:00
Zack 3890f5d306
feat: User deletion (#177)
* feat: database trigger for hard delete of user

* feat: clean delete user data

* feat: hard delete when admin deletes
2023-11-27 01:07:45 +08:00
Nathan.fooo 54ef875f5f
feat: notify user profile change (#163)
* feat: send user profile changes via ws

* test: add tests
2023-11-13 22:58:18 +08:00
Nathan.fooo 487bc25767
fix: subscriber drop (#158)
* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test
2023-11-10 13:41:12 +08:00
Nathan.fooo f1dad07caa
feat: http init sync (#151)
* fix: bugs

* chore: support message type

* refactor: ws message

* feat: post realtime message

* feat: use proto to reduce message size

* ci: update

* ci: install protoc

* ci: update docker

* chore: update ci

* ci: update docker

* ci: fix

* ci: fix

* ci: fix
2023-11-08 15:41:39 +08:00
Fu Zi Xiang 91bc8568ae
fix: unwrap 2023-11-07 11:00:37 +08:00
Fu Zi Xiang d06f8d89b7
chore: remove unneeded print 2023-11-07 10:01:00 +08:00
Fu Zi Xiang 32390676f1
fix: user not allowed 2023-11-07 01:17:08 +08:00
Fu Zi Xiang 17044ad4f0
feat: indempotency restart for email autoconfirm and non autoconfirm 2023-11-07 00:32:28 +08:00
Fu Zi Xiang c029759573
chore: merge with main 2023-11-06 23:54:36 +08:00
Fu Zi Xiang a6f7198443
chore: remove unneeded binding 2023-11-06 13:54:45 +08:00
Fu Zi Xiang b785b1a482
fix: restartable server when autoconfirm is true 2023-11-06 13:50:38 +08:00
Nathan.fooo d088fd2ec8
feat: ws frame size (#146)
* chore: update ws frame size

* chore: update

* chore: update frame size to 64 kb

* chore: fix merge

* chore: update

* chore: update collab rev

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update ws state

* chore: update pg max connect

* chore: fix crash
2023-11-04 21:42:06 +08:00
Zack 37a759ead6
feat: Request Id (#144)
* feat: added basic monitoring stack

* docs(grafana): import steps and recommended imports

* feat(metric): api metrics

* feat(prometheus): add appflowy cloud metric scraping job

* fix: put back tracing logger

* feat: middleware for path counter

* feat: added gotrue to prometheus metrics

* feat: added nginx to prometheus metrics

* feat: added latency and result to metrics

* feat(nginx): set request id to header

* feat: added request id in nginx

* feat: simplify metric collection

* feat: added simple grafana dashboard for appflowy cloud

* feat(logging): added logging stack

* docs: improve monitoring docs

* docs: added known common issues

* feat: give persistence

* feat: add exemplars

* fix: docker compose volumes

* feat(grafana): add trace id to prometheus datasource

* feat(grafana): improve dashboard to include exemplars

* feat: request id middleware

* docs: move doc to more relevant place

* chore: clean up logs

* feat: try json fmt

* feat: remove pretty

* feat: logging use json in prod but pretty in local
2023-11-03 16:57:05 +08:00
Zack 1cc1158d36
feat: Metrics (#141)
* feat: added basic monitoring stack

* docs(grafana): import steps and recommended imports

* feat(metric): api metrics

* feat(prometheus): add appflowy cloud metric scraping job

* fix: put back tracing logger

* feat: middleware for path counter

* feat: added gotrue to prometheus metrics

* feat: added nginx to prometheus metrics

* feat: added latency and result to metrics

* feat(nginx): set request id to header

* feat: added request id in nginx

* feat: simplify metric collection

* feat: added simple grafana dashboard for appflowy cloud

* feat(logging): added logging stack

* docs: improve monitoring docs

* docs: added known common issues

* feat: give persistence

* feat: add exemplars

* fix: docker compose volumes

* feat(grafana): add trace id to prometheus datasource

* feat(grafana): improve dashboard to include exemplars
2023-11-02 15:34:00 +08:00
Nathan.fooo cb9cdb9280
feat: collab permission test (#126)
* chore: check permission for collab storage proxy

* feat: add permission tests

* chore: skip log
2023-10-19 17:12:21 +08:00
Nathan.fooo 9dc7bbeee0
feat: Collab access control (#120)
* chore: check collab message with access permission

* chore: imple collab permission service

* refactor: migrations

* chore: collab member ops

* chore: collab permission

* chore: update can edit workspace collab

* chore: fix test

* feat: fetch collab members

* chore: fix test

* chore: fix client api

* chore: check permission for collab storage proxy
2023-10-17 14:00:04 +08:00
Nathan.fooo 88be0c2433
refactor: access control (#118)
* refactor: access control

* chore: fix test
2023-10-15 10:56:56 +08:00
Nathan.fooo 5c58f95f9f
feat: workspace access control (#115)
* feat: workspace access control

* feat: extact workspace id from path

* feat: create member with permission set

* chore: update sqlx files

* chore: fix cargo toml warn

* chore: clippy
2023-10-13 20:02:29 +08:00
Nathan.fooo ef726092be
refactor: file storage (#110)
* refactor: return file id after save to disk

* test: add tests

* chore: mine type

* test: disbale two client sync test

* chore: get file with url

* refactor: client api
2023-10-11 21:36:57 +08:00
Nathan.fooo a02da07627
Test deploy docker log (#105)
* chore: try to fix logs in docker

* ci: test deploy

* ci: log level
2023-10-08 21:42:25 +08:00
Nathan.fooo c0aea377d0
test: test with dev client (#91)
* test: test with dev client

* chore: add nginx conf

* chore: fix clippy

* chore: disable docker in lint
2023-10-04 21:43:21 +08:00