Build a Cross Platform FAISS Index

Building a FAISS similarity search index is not cross-platform. If you save the index locally using aarch64 and try to load it an x86_64 environment it will not work (for me, it loads an empty index). Using docker, you can save an index built locally so it’s compatible in other architectures.

To do that, run a container of the image which has your setup using the --platform flag (matching the architecture of the servers you want the index to be loaded) and run your code to build and save the index (using the FAISS save_local method). The index files generated in the container will get synced to your local machine via the volume mount so you can use them how you need (e.g. check them into version control, store them in S3).

docker run --rm -it \
       --platform linux/x86_64 \
       -v $PWD:/my/container/path myimage:latest \
       sh