mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Sanderhoff-alt
0478c09c41
Retain and import paths convert float embeddings to pgvector vector literals
for asyncpg binding (insert_facts_batch, compute_semantic_links_within_batch,
update_memory_unit_embedding).
The baseline implementation used a Python generator:
"[" + ",".join(repr(float(value)) for value in embedding) + "]"
For 500 facts (768,000 floats at 1536d), this allocated 768,000 PyFloat objects
and 768,000 PyUnicode strings, taking ~220 ms CPU time and ~15 MB heap memory.
The optimized implementation leverages np.frombuffer on PackedEmbedding
(array('f')) for zero-copy buffer views, and orjson.OPT_SERIALIZE_NUMPY to
format floats directly into the output byte buffer using Rust Ryu SIMD:
* Promotes numpy to explicit direct dependency across hindsight-api and dev;
* Formats shortest float32 representation (byte-identical Postgres storage);
* Isolates _repr_literal fallback helper for non-finite and non-float inputs;
* Unifies _dumps_or_repr_fallback with single payload parameter and no option branching;
* Streamlines embedding_to_pgvector into a concise polymorphic dispatcher;
* Seamlessly supports array('f'), list[float], tuple, ndarray, and str.
Measured on Apple Silicon via vector-serialization-bench (best of 5 repeats):
workload baseline prod speedup peak alloc
single_bge_384 (1x 384d) 0.136 ms 0.040 ms 3.4x 36K -> 13K
single_openai_1536 (1x 1536d) 0.489 ms 0.085 ms 5.8x 144K -> 50K
batch_20_gemini_768 (20x 768d) 4.580 ms 0.514 ms 8.9x 356K -> 185K
batch_200_openai_1536 (200x 1536d) 92.64 ms 9.45 ms 9.8x 6.1M -> 3.4M
batch_500_large_doc (500x 1536d) 221.78 ms 23.84 ms 9.3x 15.0M -> 8.4M
batch_200_raw_list (200x 1536d) 87.70 ms 11.48 ms 7.6x 6.1M -> 6.0M
Throughput increased from 3.3 Mfloat/s to 32.5 Mfloat/s (~9.8x speedup on
typical retain batches), with ~44% peak memory reduction on 500-fact batches.
Includes unit tests in test_packed_embeddings.py covering bit-identical float32
roundtrips, custom non-serializable objects, non-f array fallthrough, tuples,
ndarrays, and non-finites.