I used BARD to help me to get this to work. It is now working and looking at photos. Are there plans to make this true multi modal that supports video and audio streams? This is the holy grail of open source AI. I'm still playing with this and it is impressive for an open source LLM that can run on my alienware. Thanks for posting.
Attempted to Mistral working. Receive the following error. I'm on windows and have added .exe to the command lines:
kenba@DESKTOP-UU7PDAG /cygdrive/d
$ ./mistral-7b-instruct-v0.1-Q4_K_M-server.llamafile.exe
./mistral-7b-instruct-v0.1-Q4_K_M-server.llamafile.exe: line 11: /home/kenba/.ape-1.9: cannot execute binary file: Exec format error
./mistral-7b-instruct-v0.1-Q4_K_M-server.llamafile.exe: line 11: /home/kenba/.ape-1.9: No error
Few comments for Windows:
• Use the Windows Subsystem for Linux (WSL) instead of Cygwin, as it might offer better support for the executable
• Install the CUDA Toolkit from NVIDIA (https://developer.nvidia.com/cuda-toolkit) for GPU support
• Use the command-line argument '--n-gpu-layers 35' to optimize GPU usage, if applicable
I'll update the post with these additional notes.
I used BARD to help me to get this to work. It is now working and looking at photos. Are there plans to make this true multi modal that supports video and audio streams? This is the holy grail of open source AI. I'm still playing with this and it is impressive for an open source LLM that can run on my alienware. Thanks for posting.
Is this for intel, apple silicon or both based macs?
This is the list of supported platforms:
https://news.ycombinator.com/item?id=38465916
Specifically for Mac: MacOS 15.6+ (arm64 or amd64, gpu only supported on arm64)