I’m interested in hosting something like this, and I’d like to know experiences regarding this topic.
The main reason to host this for privacy reasons and also to integrate my own PKM data (markdown files, mainly).
Feel free to recommend me videos, articles, other Lemmy communities, etc.
There’s a local llama subreddit with a lot of good information and 4chan’s /g/ board will usually have a good thread with a ton of helpful links in the first post. Don’t think there’s anything on lemmy yet. You can run some good models on a decent home pc but training and fine tuning will likely require renting out some cloud gpus.