The Hobbyist

Congrats! Amazing project, exciting interface and you went the extra mile on the integration side with third parties. Kudos!

Edit: I’ll definitely have to try it out!

Perhaps give Ramalama a try?

https://github.com/containers/ramalama

Indeed, Ollama is going a shady route. https://github.com/ggml-org/llama.cpp/pull/11016#issuecomment-2599740463

I started playing with Ramalama (the name is a mouthful) and it works great. There is one or two more steps in the setup but I’ve achieved great performance and the project is making good use of standards (OCI, jinja, unmodified llama.cpp, from what I understand).

Go and check it out, they are compatible with models from HF and Ollama too.

https://github.com/containers/ramalama

I will always remember my first experience using MacOS: I am comfortable with computers and a relative needed help with their recently purchased macbook. I had plugged in a USB stick to transfer some files and was done and wanted to eject it. I spent way too much time than I care to admit, trying all possible options, right-clicks, settings, everything imaginable, to eject the damn thing.

It was impossible to me to find the simplest operation with a USB stick, something required to operate it. I capitulated and looked online. The solution? I had to drag and drop the USB stick icon into the trashcan!?!?!?

To this day, I will never understand the absolute ridicule of this and I will never comprehend how anyone is expected to figure it out on their own. And this is from the OS touted as the most user friendly and intuitive. Go figure.

Edit: this was a long while back, no idea how it is nowadays.

Would you be able to share more info? I remember reading their issues with docker, but I don’t recall reading about whether or what they switched to. What is it now?

Regarding photos, and videos specifically:

I know you said you are starting with selfhosting so your question was focusing on that, but I would like to also share my experience with ente which has been working beautifully for my family, partner and myself. They are truly end to end encrypted, with the source code available on github.

They have reasonable prices. If you feel adventurous you can actually also host it yourself. They have advanced search features and face recognition which all run on device (since they can’t access your data) and it works very well. They have great sharing and collaborating features and don’t lock features behind accounts so you can actually gather memories from people on your quota by just sharing a link. You can also have a shared family plan.

From what I understand, sealed sender is implemented on the client side. And that’s what’s in the github repo.

They have to know who the message needs to go to, granted. But they don’t have to know who the message comes from, hence why the sealed sender technique works. The recipient verifies the message via the keys that are exchanged if they have been communicating with that correspondent before or else it is a new message request.

So I don’t see how they can build social graphs if they don’t know who the sender if all messages are, they can only plot recipients which is not enough.

Signal absolutely can does provide social graphs, message frequency, message times, message size.

Do you have anything to back this up?

and requires phone numbers (meaning your real identity in the US).

This gets shared a lot as a major concern for all services requiring a phone number. It is definitely true that by definition, a phone number is linked to a person’s identity, but in the case of signal, no other information can be derived from it. When the US government requests data for that phone number from Signal, like they occasionally do, the only information Signal provides them with is whether they do have a signal account and when they registered it last and when they last signed in. How is that truly problematic? For all other services which require a phone number, you would have much more information which is where it is truly problematic, say social graph, text messages, media, locations, devices etc. But none of that is accessible by Signal. So literally the only thing signal can say is whether the person has an account, that’s about it. What’s the big deal about it? Clearly the US government already has your phone number because they need it to make the request for Signal, but they gain absolutely no other information.

Ollama is very useful but also rather barebones. I recommend installing Open-Webui to manage models and conversations. It will also be useful if you want to tweak more advanced settings like system prompts, seed, temperature and others.

You can install open-webui using docker or just pip, which is enough if you only care about serving yourself.

Edit: open-webui also renders markdown, which makes formatting and reading much more appealing and useful.

Edit2: you can also plug ollama into continue.dev, an extension to vscode which brings the LLM capabilities to your IDE.

The interface called open-webui can run in a container, but ollama runs as a service on your system, from my understanding.

The models are local and only answer queries by default. It all happens on the system without any additional tools. Now, if you want to give them internet access, you can, it is an option you have to setup and open-webui makes that possible though I have not tried it myself. I just see it.

I have never heard of any llm “answer base queries offline before contacting their provider for support”. It’s almost impossible for the LLM to do it by itself without you setting things up for it that way.

whats great is that with ollama and webui, you can as easily run it all on one computer locally using the open-webui pip package or in a remote server using the container version of open-webui.

Ive run both and the webui is really well done. It offers a number of advanced options, like the system prompt but also memory features, documents for RAG and even a built in python ide for when you want to execute python functions. You can even enable web browsing for your model.

I’m personally very pleased with open-webui and ollama and they both work wonders together. Hoghly recommend it! And the latest llama3.1 (in 8 and 70B variants) and llama3.2 (in 1 and 3B variants) work very well, even on CPU only, for the latter! Give it a shot, it is so easy to set up :)