Commit Graph

291 Commits

Author SHA1 Message Date
Nathan.fooo 5c1a16cec5
chore: Add sync test (#225)
* chore: resume old tests

* chore: bump collab commit id

* chore: clippy

* chore: fix audit
2023-12-19 02:56:56 +08:00
Nathan.fooo 69ed6ff4e8
feat: create snapshot for collab (#221) 2023-12-17 13:38:55 +08:00
Nathan.fooo 1eed034a83
ci: docker build (#219) 2023-12-17 10:55:10 +08:00
Nathan.fooo b39621e389
refactor: reduce select all workspace memebers call (#216)
* refactor: reduce select all workspace memebers call

* chore: docker build

* refactor: move tests

* chore: expose enforcer on tests

* refactor: update workspace member when recv pg notification

* chore: update test

* chore: commit sqlx file
2023-12-17 02:46:05 +08:00
Nathan.fooo f4c87f1f75
chore: bump collab rev (#213) 2023-12-15 11:30:15 +08:00
Zack b638503467
Update Cargo.toml
Co-authored-by: JesseBot <jessebot@linux.com>
2023-12-13 23:59:55 +08:00
Zack 7d1461232f
Update Cargo.toml
Co-authored-by: JesseBot <jessebot@linux.com>
2023-12-13 23:59:42 +08:00
jessebot 26d1aa2e6e fix: turn on ssl features for rust-s3 2023-12-13 11:41:14 +01:00
Nathan.fooo 9589054f38
refactor: sync protocol (#203)
* refactor: sync protocol

* chore: update collab rev
2023-12-11 11:27:11 +08:00
Jiraffe7 36ef0f13b0
feat: casbin access control (#178)
* feat: casbin for access control

* fix: method to generate database url instead of ToString

* fix: hold write lock when modifying policies

* chore: fix compile

* fix: remove db constraint for lib integration tests

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-12-11 11:08:15 +08:00
Nathan.fooo f13a03bead
chore: bump up yrs (#193) 2023-12-04 12:34:37 +08:00
Nathan.fooo 0e2ec381df
chore: implement symmetric encrypt (#192) 2023-12-04 11:50:56 +08:00
Nathan.fooo 58ffae16b1
feat: enable create default documents for user when sign up (#184)
* chore: send collab with interval

* feat: create workspace template
2023-11-29 06:11:20 +08:00
Nathan.fooo 82bcb5e879
feat: Deployment update (#169)
* docs: add ubuntu setup guide

* feat: add docker ci

* feat: create admin frontend image

* feat: trigger push docker ci

* ci: multiple arch

* fix: build

* chore: update ci

* chore: update ci

* fix: build

* fix: build

* fix: test

* fix: database access

* fix: test

* fix: test

* chore: update

* chore: update
2023-11-16 00:29:18 +08:00
Nathan.fooo 147317bd1c
chore: update docs (#165)
* chore: update docs

* chore: update

* chore: update
2023-11-14 23:44:51 +08:00
Nathan.fooo 79a02edd00
feat: notify user via websocket after profile changed (#164)
* feat: notify user via websocket after profile changed

* chore: update

* chore: update
2023-11-14 12:47:10 +08:00
Nathan.fooo 29b39f9ba3
feat: Client api url (#162)
* chore: expose client base url
2023-11-13 20:43:04 +08:00
Nathan.fooo 2e14dcf129
feat: Client api with test util feature (#161)
* chore: expose client base url

* feat: add test util feature

* ci: fix af cloud test

* ci: fix af cloud test
2023-11-12 19:44:49 +08:00
Nathan.fooo c7e90eb24b
feat: stop recv messages when subscribe drop (#159)
* feat: stop recv messages when subscribe drop

* chore: update collab rev

* chore: fmt
2023-11-11 12:04:52 +08:00
Nathan.fooo 487bc25767
fix: subscriber drop (#158)
* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test

* chore: test
2023-11-10 13:41:12 +08:00
Nathan.fooo f1dad07caa
feat: http init sync (#151)
* fix: bugs

* chore: support message type

* refactor: ws message

* feat: post realtime message

* feat: use proto to reduce message size

* ci: update

* ci: install protoc

* ci: update docker

* chore: update ci

* ci: update docker

* ci: fix

* ci: fix

* ci: fix
2023-11-08 15:41:39 +08:00
Nathan.fooo 41e01b91fc
feat: update collab rev and fix some bugs (#148)
* chore: remove todo

* chore: update collab rev

* chore: fmt

* chore: fmt

* chore: cache collab user

* fix: internal error

* ci: fix test

* chore: update collab rev

* ci: fix
2023-11-05 22:43:20 +08:00
Nathan.fooo 5559e8cbd7
chore: update logs (#147)
* chore: update logs

* chore: update logs

* chore: update logs

* chore: update logs
2023-11-05 13:25:47 +08:00
Nathan.fooo d088fd2ec8
feat: ws frame size (#146)
* chore: update ws frame size

* chore: update

* chore: update frame size to 64 kb

* chore: fix merge

* chore: update

* chore: update collab rev

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update ws state

* chore: update pg max connect

* chore: fix crash
2023-11-04 21:42:06 +08:00
Nathan.fooo 937e3bd9b2
fix: fix some bugs (#145)
* chore: update

* chore: pub error code

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update max frame size

* chore: update max frame size

* chore: ws buffer size

* chore: update
2023-11-03 14:15:08 +08:00
Nathan.fooo 9c911a3a94
refactor: Error (#143)
* chore: remove api

* chore: update log

* refactor: error

* chore: fmt

* ci: fix test

* ci: fix

* chore: remove unstable
2023-11-02 17:17:34 +08:00
Zack 1cc1158d36
feat: Metrics (#141)
* feat: added basic monitoring stack

* docs(grafana): import steps and recommended imports

* feat(metric): api metrics

* feat(prometheus): add appflowy cloud metric scraping job

* fix: put back tracing logger

* feat: middleware for path counter

* feat: added gotrue to prometheus metrics

* feat: added nginx to prometheus metrics

* feat: added latency and result to metrics

* feat(nginx): set request id to header

* feat: added request id in nginx

* feat: simplify metric collection

* feat: added simple grafana dashboard for appflowy cloud

* feat(logging): added logging stack

* docs: improve monitoring docs

* docs: added known common issues

* feat: give persistence

* feat: add exemplars

* fix: docker compose volumes

* feat(grafana): add trace id to prometheus datasource

* feat(grafana): improve dashboard to include exemplars
2023-11-02 15:34:00 +08:00
Nathan.fooo 1aba1f0cf4
feat: workspace usage test (#129)
* feat: workspace usage test

* test: add tests
2023-10-22 17:52:10 +08:00
Nathan.fooo 9dc7bbeee0
feat: Collab access control (#120)
* chore: check collab message with access permission

* chore: imple collab permission service

* refactor: migrations

* chore: collab member ops

* chore: collab permission

* chore: update can edit workspace collab

* chore: fix test

* feat: fetch collab members

* chore: fix test

* chore: fix client api

* chore: check permission for collab storage proxy
2023-10-17 14:00:04 +08:00
Nathan.fooo 5c58f95f9f
feat: workspace access control (#115)
* feat: workspace access control

* feat: extact workspace id from path

* feat: create member with permission set

* chore: update sqlx files

* chore: fix cargo toml warn

* chore: clippy
2023-10-13 20:02:29 +08:00
Fu Zi Xiang 591b66b73e
feat: auto refresh session 2023-10-13 12:08:53 +08:00
Nathan.fooo 3611c2076e
feat: update ws connect setting (#111)
* chore: update ws connect setting

* chore: remove token when token is invalid

* chore: close the ws connect when overflow happened

* fix: reset timing

* fix: reset timing

* chore: sleep if timeout not reach

* chore: sleep if timeout not reach
2023-10-12 16:51:44 +08:00
Zack 8023efae97
feat: add admin frontend project (#102)
* feat: add admin frontend project

* feat: session token WIP

* feat: session token WIP - 2

* feat: session token and dev ease

* feat: added logout

* extractor for access token

* feat: added funtionality to show all users

* feat: show user list on frontend

* feat: user details

* fix: remove unneeded print statements

* chore: enable back to docker-compose.yml

* fix: integration tests

* feat: added redis session

* feat: change to session management using redis

* feat: add email to display at home page

* feat: added logging, logout

* feat: use redirect temporary

* feat: added add user by email

* feat: added add user set password web ui

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-10-11 22:17:41 +08:00
Nathan.fooo 00a086bc40
feat: collab message definition (#109)
* feat: collab message definition

* chore: update collab rev

* chore: maximum payload size
2023-10-10 17:45:44 +08:00
Nathan.fooo ad5b5b631d
feat: Set token from external (#107)
* chore: set token

* chore: set token

* chore: update

* chore: rm log
2023-10-09 23:07:34 +08:00
Nathan.fooo 089b3046ab
chore: implement batch get (#106)
* chore: implement batch get

* chore: add request id and update the local_server.sh

* chore: update collab commit id
2023-10-08 23:53:16 +08:00
Nathan.fooo a02da07627
Test deploy docker log (#105)
* chore: try to fix logs in docker

* ci: test deploy

* ci: log level
2023-10-08 21:42:25 +08:00
Zack 583f82b7fb
refactor: collab http (#89)
* refactor: collab http

* fix: sqlx uuid in types client_api
2023-10-04 16:59:03 +08:00
Nathan.fooo cf84557ebe
feat: oauth provider (#86)
* chore: remove opener

* chore: rename method

* chore: add docs

* chore: modify env
2023-10-03 22:06:07 +08:00
Nathan.fooo b0c213b5c0
fix: message queue (#85)
* fix: message queue

* chore: retry sink collab message

* chore: update
2023-10-02 15:28:35 +08:00
Nathan.fooo 3133e5ec32
chore: try fix test (#84) 2023-09-30 12:36:42 +08:00
Nathan.fooo 19ee96db4a
chore: rename crate (#82) 2023-09-28 22:20:48 +08:00
Zack a6eb8607c9
feat: async read for file upload (#80)
* feat: async read for file upload

* feat: async read for file download

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-28 14:53:41 +08:00
Nathan.fooo b8bd53ff5c
feat: create collab sync crate (#79)
* feat: create collab sync crate

* test: add tst

* test: add test

* test: update tests

* chore: clippy

* chore: clippy

* chore: clippy

* chore: add retry when subscribe group

* chore: disable realtime test
2023-09-28 09:28:46 +08:00
Zack ff6a8e1eaf
feat: File api (#70)
* feat: s3 minio wip

* feat: s3 minio bucket create idempotent

* feat: put storage setting into configurations

* chore: clippy lint

* feat: add setting to base config

* feat: add configuration for prod

* fix: allow use different minio host

* feat: add server file storage

* fix: add missing file

* feat: add code template

* feat: add http api

* feat: added file upload to client api

* feat: database metadata impl

* fix: added missing files

* feat: added test cases and improve impl

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-27 10:21:37 +08:00
Nathan.fooo b729e3529d
chore: bump collab version (#78)
* chore: bump collab version

* chore: bump collab version

* chore: update

* chore: update

* chore: update

* chore: update tests

* chore: clippy

* chore: clippy

* chore: clippy
2023-09-26 22:52:41 +08:00
Nathan.fooo b08a4a9016
chore: expost sign in url (#76)
* chore: expost sign in url fc

* refactor: realtime test

* chore: add logs

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update
2023-09-26 09:26:35 +08:00
Nathan.fooo 0883ae94b8
test: sign in url test (#75)
* test: sign in url test

* fix: Tungstenite allows remote attackers to cause a denial of service

* chore: enable feature

* chore: update
2023-09-24 13:03:24 +08:00
Zack e3ff765137
feat: Gotrue admin api (#74)
* feat: set up admin account and auto confirm during appflowy start

* feat: client auth against gotrue whenever possible

* feat: admin add user

* feat: implement admin add user

* feat: generate registered user

* fix: enable cloud feature for client_api

* fix: test same user fix

* feat: added admin magic link generation
2023-09-23 20:06:37 +08:00
Nathan.fooo cbae9493b7
chore: update collab commit id (#71) 2023-09-22 23:00:36 +08:00
Zack 67706f9e8e
feat: File api (#68)
* feat: s3 minio wip

* feat: s3 minio bucket create idempotent

* feat: put storage setting into configurations

* chore: clippy lint

* feat: add setting to base config

* feat: add configuration for prod

* fix: allow use different minio host
2023-09-22 15:10:41 +08:00
Zack 0676db7ed6
feat: Workspace members (#63)
* fix: create user if they sign up with oauth

* feat: add workspaces_members add and remove operations

* feat: add insert workspace members

* feat: attach db ops to api ops

* refactor: separate user and workspace scope

* chore: add sqx query metadata

* chore: add 2nd registered user for testing

* ci: use valid email

* ci: fix registered password invalid

* fix: use email instead of uid

* fix: use email instead of uid sqlx query metadata

* feat: add test for workspace member

* test: added workspace add and delete with checking

* ci: fix email
2023-09-19 20:59:16 +08:00
Nathan.fooo e9fc2af43c
feat: implement websocket client (#47)
* feat: implement websocket client

* chore: clippy

* chore: update

* chore: update env
2023-09-15 12:11:04 +08:00
Nathan.fooo 939ea29c3b
feat: Realtime multiple conn test(#45)
* chore: multiple connection with same user

* chore: connect with device id

* fix: multiple connection

* test: add more test

* chore: fmt
2023-09-15 09:59:37 +08:00
Nathan.fooo e03a6ce587
test: fix collab edit test (#42)
* test: fix

* chore: cargo sqlx check workspce

* chore: remove rocksdb deps
2023-09-14 14:05:15 +08:00
Nathan.fooo b1adcc7919
test: websocket connect and then edit collab test (#40)
* refactor: realtime dir

* test: add test

* test: fix
2023-09-14 09:34:24 +08:00
Nathan.fooo c42158b7cb
test: add ws connect test (#39)
* test: add ws connect test

* chore: format log
2023-09-13 17:04:42 +08:00
Nathan.fooo 3aa4d20ad3
chore: reduce deps of client_api crate (#37) 2023-09-13 13:12:07 +08:00
Nathan.fooo b32fcff4d4
test: Collab handler test (#36)
* chore: implement collab requests

* test: add more tests

* chore: update scope path
2023-09-13 12:52:22 +08:00
Zack 0d59211e55
feat: expose workspace and profile api (#32)
* feat: expose workspace and profile api

* feat: add impl for client_api

* feat: add test case for workspace and profile
2023-09-12 16:14:14 +08:00
Nathan.fooo 9ac53dca8e
chore: implement collab storage using postrges (#31)
* chore: implement collab storage using postrges

* chore: update collab rev

* chore: update .sqlx files
2023-09-12 15:36:33 +08:00
Zack 853f089ca0
feat: test gotrue when app start, use state for gotrue client, use go… (#24)
* feat: test gotrue when app start, use state for gotrue client, use gotrue UUID instead of BIGINT for user uid

* feat: backward compatibility with native authetication

* fix: native auth use uuid instead

* fix: sqlx preparation

* fix: sqlx metadata and unused variable

* fix: clippy

* fix: create workspace if not exist after sign in

* chore: update .sqlx with new query

* feat: query workspace

* revert: revert using i64 for user id

* fix: update sqlx queries

* fix: update .sqlx metadata

* feat: only use uuid from sqlx types, select user_profile view

* ci: add version check

* ci: add missing sqlx files

* chore: cargo fmt
2023-09-11 17:16:41 +08:00
Nathan.fooo d0fdde7bf5
chore: remove unused deps (#29)
* chore: remove unused deps

* tests: fix
2023-09-11 16:21:16 +08:00
Nathan.fooo 63c992602f
tests: fix realtime tests (#27)
* tests: fix realtime tests

* chore: update collab rev

* ci: fix realtime ci

* ci: update test path
2023-09-11 15:14:58 +08:00
Zack d4577f5e61
Feat: enrich http dto (#20)
* feat: add scripts for easier dev

* feat: add Json<T> as resp type for http server

* feat: restructure server and client shared entity

* feat: restructure server and client shared entity

* feat: use serde_repr to deserialize enum into json number

* feat: minor adjustment to results and server error

* feat: test: fix unconfirmed email message

* feat: test: add test case for existing user

* feat: test: modified api for sign_out

* feat: test: add test case pass
2023-09-08 20:01:28 +08:00
Nathan.fooo 384501a00d
refactor: save collab data in pg (#19) 2023-09-08 19:41:35 +08:00
Nathan.fooo c1f8d79169
chore: separate crates (#18) 2023-09-07 19:40:11 +08:00
Zack 36499be26c
docs: test + deploy + environmental variables (#14)
* doc: test + deploy + environmental variables

* doc: local dev and testing(WIP)

* dev: set up local testing

* fix: ci: add DATABASE_URL env

* chore: update .env

* ci: fix

* dev: docker-compose-dev.yml

* ci: update

* ci: test env

* ci: add services

* ci: add services

* ci: update port

* ci: rename ci.yaml to rustlint.yaml

* ci: test

* ci: simplify tests

* ci: use cargo sqlx commands directly

* ci: fix: registered user

* ci: fix: test registered user creation

* ci: fix: test registered user creation - 2

* ci: fix: debug docker ci

* ci: fix: docker integration registered user

* ci: fix: docker integration registered user

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-07 15:29:39 +08:00
Nathan.fooo f9825d4221
feat: refactor server collab (#15)
* refactor: collab storage

* chore: fix test

* chore: update sqlx version

* chore: disable realtime test

* chore: sqlx prepare

* chore: remove init.sql
2023-09-06 18:22:40 +08:00
Zack c8ae099e90
feat: Auth server integration (#9)
* feat: integrate signup with gotrue auth server

* feat: added email and password sign in via auth server

* feat: added sign in scenarios and client handling

* fix: added missing files

* feat: refine api and store access token in client

* feat: impl token decode and placeholder for signout

* feat: added impl for logout with auth server

* feat: auth server dependency injection, client sign_out impl

* feat: abstract verify token, add placeholder for update endpoint

* feat: make authorization as a param of http handler request

* feat: add update_user to client of auth server

* feat: add client code and test cases for password change

* chore: rename appflowy-server to appflowy-cloud

* ci: upgrade reqwest version to fix audit issue

* ci: fix again

* ci: added gotrue env secrets

---------

Co-authored-by: nathan <nathan@appflowy.io>
2023-09-04 11:23:40 +08:00
Zack d50de4e4d8
feat: added capability to decode supabase web token (#8) 2023-08-11 08:43:21 +08:00
Zack 58f0cdd235
feat: Client API and Integration test (#7)
* feat: change to client from test_server

* feat: change to client from test_server: register_success

* fix: use time in email for indempotency

* added rest of register test case

* feat: added client impl for login

* feat: Client: store logged token

* feat: added password change impl

* feat: remove test api, move to using Client

* chore: remove unneeded code

* chore: add client's integration test

* chore: try fix github workflow

* chore: try fix github workflow

* chore: try fix github actions - 2

* chore: try fix github actions - change rust toolchain

* chore: try fix github actions - remove name field

* chore: ci: fixed missing run key

* chore: ci: fixed missing Checkout

* chore: ci: added env settings for docker-compose

* chore: ci: added env BACKEND_VERSION for docker-compose

* chore: ci: added context . to docker-compose.yml

* chore: ci: added old dockerfiles to archive, default values in docker-compose.yml

* chore: github workflow: build server with env SQLX_OFFLINE true

* chore: remove unneeded deps

* chore: ci: try SQLX_OFFLINE export

* fix: ci: remove unneeded integration test ci, spawn server while cargo test

* fix: lint and github workflow

* fix: lint

* fix: github workflows: use workspaces instead of key

* fix: lint: use as_deref instead
2023-08-10 09:49:47 +08:00
Nathan.fooo e65b6333b1
Feat: multi ws conn (#6)
* feat: multi conn

* fix: multi connect
2023-08-08 16:13:18 +08:00
Nathan.fooo 8b9e6584d2
feat: ws reconnect (#5)
* feat: ws reconnect

* chore: update collab rev
2023-05-10 20:54:10 +08:00
Nathan.fooo 90ae1d5fb6
Feat/ws test (#4)
* test: ws test

* test: update

* test: update

* test: sync update

* feat: ws test
2023-05-10 20:26:30 +08:00
Nathan.fooo 18e950a829
feat: ws connect (#3)
* chore: ws

* chore: build client stream

* feat: test ws connect

* ci: fix ci
2023-05-08 19:03:50 +08:00
nathan 920e7f9564 ci: fix audit 2023-03-16 21:33:11 +08:00
nathan 648ec41cf3 ci: fix warning 2023-03-16 21:16:46 +08:00
appflowy 0fa6536c7a refactor: use i64 as user id 2023-03-16 11:23:42 +08:00
nathan 277b54711c feat: add snowflake 2023-03-16 09:36:49 +08:00
appflowy 6110c69374 feat: revdb 2023-03-15 17:21:18 +08:00
appflowy faacd8dd47 feat: create token crate 2023-03-15 14:41:23 +08:00
nathan bbcfe33fe7 fix: request fail caused by certificate issue 2023-03-15 09:32:23 +08:00
nathan 4adfd1a75f ci: fix test 2023-03-14 14:45:53 +08:00
nathan 5a7e223b99 feat: support ws 2023-03-14 00:05:23 +08:00
nathan 25b1f8eeb7 feat: change password 2023-03-13 09:17:14 +08:00
nathan 508961d07d feat: register user test 2023-03-12 16:37:54 +08:00
nathan 690ea42d85 refactor: config log 2023-03-12 09:18:06 +08:00
nathan 7b706b54ce refactor: auth 2023-03-11 22:01:25 +08:00
appflowy 3012acce30 refactor: use cargo workspace 2022-02-07 15:08:49 +08:00
appflowy 0bda429d93 init appflowy server 2022-02-06 10:55:48 +08:00