Commit Graph

18 Commits

Author SHA1 Message Date
Nathan.fooo 1614474c2d
chore: check collab type before index (#1093) 2024-12-19 22:03:10 +08:00
Nathan.fooo abf827f2a6
chore: query multiple collab embedding state (#1081)
* chore: query multiple collab embedding state

* chore: clippy
2024-12-17 23:23:43 +08:00
Nathan.fooo ec9c38254b
chore: open ai call directly (#1074)
* chore: use open ai embedding

* chore: update test

* chore: retry

* chore: metrics

* chore: clippy

* chore: env

* chore: delete test
2024-12-15 23:29:46 +08:00
Nathan.fooo 2ab1e94b99
chore: create embed in background (#1071)
* refactor: create indexer scheduler

* refactor: replace current calls

* chore: use rayon thread pool

* chore: index unindexed collab using thread pool

* chore: index collab when write to disk

* chore: add test

* chore: add test

* chore: add test

* chore: fix client api deps

* chore: chat with document test

* chore: fix test

* chore: clippy

* chore: update test
2024-12-15 12:29:51 +08:00
Nathan.fooo af38efe6d9
feat: sync document through http request (#1064)
* chore: query embedding

* chore: create embeddings

* chore: apply update to editing collab

* refactor: web-update

* chore: calculate missing update the sv is not none

* chore: add test

* chore: fix audit

* chore: commit sqlx

* chore: fix client api

* test: add

* chore: clippy

* chore: fix collab drop when save
2024-12-12 14:53:07 +08:00
Bartosz Sypytkowski cdf98d1a83
chore: add logging for reindexing docs after restart (#1056) 2024-12-11 17:17:29 +08:00
Bartosz Sypytkowski 4fb6df5c14
chore: reduce number of SQL statements executed when inserting/updating collab (#1022)
* chore: reduce number of SQL statements executed when inserting new collab

* chore: add workspace id check

* chore: fix previous query

* chore: upsert collabs and embddings as stored procedures

* chore: fix clippy errors

* chore: fix stored procedure to compare workspaces correctly
2024-11-27 13:00:00 +08:00
Nathan.fooo 655f13bc27
chore: search and chat (#999)
* chore: add test for search and chat

* chore: update test

* chore: update test

* chore: update ci

* chore: fix security audio

* chore: multiple core docker build

* chore: multiple core docker build

* chore: update ci

* chore: update model setting

* chore: test ci

* chore: use tiktoken to calcualte token length

* chore: remove env

* chore: use spawn_blocking with condition

* chore: docs

* chore: clippy

* chore: clippy

* chore: docker logs

* chore: pass message id

* chore: clippy
2024-11-16 14:52:12 +08:00
Nathan.fooo 5d356718fa
fix: using spawn_blocking in order to not block the runtime (#793)
* fix: using blocking in order to not block the runtime

* chore: increase deps count
2024-09-05 09:19:34 +08:00
Nathan.fooo 365c64c5c2
chore: use colla storage instead of select from db directly (#782) 2024-09-02 21:25:21 +08:00
Nathan.fooo c2a5369313
chore: set chat request timeout (#777)
* chore: set chat request timeout

* chore: clippy
2024-09-01 15:16:11 +08:00
Bartosz Sypytkowski 8beac5c85f
chore: reuse group collab for indexing (#737)
* chore: reuse group collab for indexing

* chore: fix linting errors

* chore: post rebase fixes
2024-08-28 00:13:41 +08:00
nathan c77d0f9c2c chore: spawn indexer 2024-08-26 09:47:46 +08:00
nathan b4d79b954b chore: update logs 2024-08-23 10:30:52 +08:00
Bartosz Sypytkowski c27866fb25 chore: upgrade code to use asynchronous locks 2024-08-13 19:59:46 +02:00
Bartosz Sypytkowski cce52a5185 chore: do not throw errors on invalid document schema during indexing 2024-06-28 05:40:44 +02:00
Bartosz Sypytkowski 5d3574d643 chore: save collab embeddings on collab create 2024-06-28 05:40:44 +02:00
Bartosz Sypytkowski f13b9ee787 chore: add indexing of collabs at application start 2024-06-28 05:40:44 +02:00