Commit Graph

224 Commits

Author SHA1 Message Date
Bartosz Sypytkowski d6317813d7 chore: allow dead code 2024-11-20 09:06:00 +01:00
Bartosz Sypytkowski b91a5fac8e chore: post merge fixes 2024-11-20 08:56:16 +01:00
Bartosz Sypytkowski d0f9059f3f Merge branch 'main' into stateless 2024-11-20 08:36:28 +01:00
Nathan.fooo 1e18180e9d
chore: set max import zip file size (#1011)
* chore: set max import zip file size

* chore: fix test
2024-11-20 14:07:36 +08:00
Nathan.fooo fd98351473
chore: adjust redis env (#1003) 2024-11-17 14:26:59 +08:00
Bartosz Sypytkowski 3799966f12
chore: store pending collab writes in memory (#1000)
* chore: write immediatelly actually writes immediatelly

* chore: fix clippy errors

* chore: add metrics to new storage queue impl

* chore: set collab batch write capacity to the same as on main branch
2024-11-17 13:25:42 +08:00
Nathan.fooo d798c81ba4
chore: support split by text len (#1002)
* chore: support split by text len

* chore: update docs

* chore: update tests
2024-11-17 12:45:20 +08:00
Nathan.fooo dcbc84dacc
chore: recreate group if it isn't exist (#1001) 2024-11-17 12:14:50 +08:00
Nathan.fooo 655f13bc27
chore: search and chat (#999)
* chore: add test for search and chat

* chore: update test

* chore: update test

* chore: update ci

* chore: fix security audio

* chore: multiple core docker build

* chore: multiple core docker build

* chore: update ci

* chore: update model setting

* chore: test ci

* chore: use tiktoken to calcualte token length

* chore: remove env

* chore: use spawn_blocking with condition

* chore: docs

* chore: clippy

* chore: clippy

* chore: docker logs

* chore: pass message id

* chore: clippy
2024-11-16 14:52:12 +08:00
Bartosz Sypytkowski 06b7ce0a29 chore: post merge fixes 2024-11-16 07:33:18 +01:00
Bartosz Sypytkowski 401cc3ad25 Merge branch 'main' into stateless 2024-11-16 07:22:34 +01:00
Khor Shu Heng d0c212ff10
refactor: remove enforcer group (#948) 2024-11-15 14:08:12 +08:00
nathan b10a0e7829 chore: fix typo 2024-11-15 13:45:25 +08:00
Nathan.fooo 97f9ff3dd8
fix: byte index 8000 is not a char boundary (#995)
* chore: fix split text boundary error and add related tests

* chore: reduce clone
2024-11-15 13:38:08 +08:00
Zack d8075a9368
Merge pull request #994 from AppFlowy-IO/fix/smtp-email-username
feat: separate smtp username from email
2024-11-14 22:00:43 +08:00
Zack Fu Zi Xiang 4b2eb1b1ba
feat: separate smtp username from email 2024-11-14 16:00:15 +08:00
Nathan.fooo 06a6ea2985
chore: retry upload file when get s3 service error (#993)
* chore: retry upload file when get s3 service error

* chore: ingore upload payload error
2024-11-14 14:39:36 +08:00
Bartosz Sypytkowski f6fef9918b
chore: split document content into multiple embeddings (#990)
* chore: split document content into multiple embeddings

* chore: fallback to uuid v4 generation due to unmaintained transient dependency

* chore: fix lint errors
2024-11-14 14:15:52 +08:00
Nathan.fooo 50d519c4ea
fix: upload file zip file expire (#987) 2024-11-13 14:57:27 +08:00
Nathan.fooo 91b0c50722
chore: remove logs (#986) 2024-11-13 13:59:23 +08:00
Nathan.fooo 9778843746
chore: disable history service (#985)
* chore: disable history service

* chore: fix build
2024-11-13 10:46:15 +08:00
Nathan.fooo 14adf250a8
chore: update ai api endpoint & rename structs (#953)
* chore: revmap loader type

* chore: revamp context

* chore: fix test

* chore: remove unused handler

* chore: create new chat api endpoint

* chore: add docs

* chore: clippy

* chore: fix test
2024-11-11 20:10:46 +08:00
Bartosz Sypytkowski eb54947ab7 chore: add config option for redis stream group workers count 2024-11-08 04:39:44 +01:00
Bartosz Sypytkowski b9ca480143 chore: add retry limit to which collab snapshot can be made 2024-11-07 13:12:17 +01:00
Bartosz Sypytkowski bc49d73b40 chore: split redis stream readers to active and one shot 2024-11-07 11:43:19 +01:00
Bartosz Sypytkowski 2d9a024554 chore: use redis stream router 2024-11-07 07:18:20 +01:00
Bartosz Sypytkowski f3cd597832 chore: fix clippy errors 2024-11-06 10:57:32 +01:00
Bartosz Sypytkowski 3fc49a8b21 chore: turn off collab preindexing 2024-11-06 10:10:55 +01:00
Bartosz Sypytkowski 863e405263 chore: add more precise logging on get unindexed collabs 2024-11-06 08:04:07 +01:00
Bartosz Sypytkowski 99203cc7f3 chore: fix flaky tests 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski 25f0461389 chore: added metrics for new stateless operations 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski e86c9de316 chore: when saving collab snapshot, do not load it unless there are pending updates in redis 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski 961c482995 chore: prevent hot loops in collab updates stream 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski 2511bd720d chore: fix clippy errors 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski ce793fa03e chore: make collab group prune grace period configurable 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski 395424bfd2 chore: detect and re-request missing updates on the server side 2024-11-05 07:48:14 +01:00
Bartosz Sypytkowski a99f986270 chore: removed appflowy history 2024-11-05 07:48:12 +01:00
Bartosz Sypytkowski 81dc66fa7b chore: fix clippy warnings 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski f684d3623d chore: fixed collab ack 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski fc1a7d28e2 chore: better stress test scenario 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski de4016659e chore: add break in cancellation of snapshot task 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski d624cdf49b chore: turn off separate appflowy history process 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski a2d3d7feb5 chore: add perf tests 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski 279a599b66 chore: fix early return from redis collab update stream reader 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski 85e25f887c chore: don't try to prune empty redis stream 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski d0b7c7d8e3 chore: update stream - live and current queries 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski 61820d7226 chore: more traces in collab group 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski a1546909c3 chore: remove server sync protocol (not used anymore) 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski 00f5a5bb7e chore: code cleanup 2024-11-05 07:47:17 +01:00
Bartosz Sypytkowski 3e612e2db6 chore: create method for receiving the most recent snapshot 2024-11-05 07:47:17 +01:00