ApurvNS

I'm a software engineer and have experience shipping projects that utilize AI models via APIs. However, I had no direct experience with the specifics of training models. My goal was to take a local LLM and have it answer questions based on a custom dataset, a common task that turned out to have more nuance than I initially expected. This post documents my process, the initial incorrect assumptions I made, and the working RAG (Retrieval-Augmented Generation) pipeline I ended up with.

Note : I haven't written the code presented in this post myself. The entire process was an exercise in prompting; the scripts and conclusions are the result of asking an LLM, like ChatGPT, a series of basic questions.

Attempt 1: Context Stuffing with Ollama's Modelfile

My first attempt involved Ollama. I found a Python script that appeared to "fine-tune" a model by taking text files as input. I ran it with a few documents, and it quickly produced a new model that could answer questions...

In design: remain simple... In thought: embrace depth. In relations: be kind. In leading: be just. In work: be competent. In action: consider timing.

When the work is done log off and etach. This is the way of heaven.

Thirty spokes join the hub, but the wheel’s usefulness depends on the emptiness at the center.

Clay forms a vessel, but its capacity to contain depends on the emptiness inside.

We build walls with windows and doors, but the utility of the room depends on the emptiness within.

Therefore, We enjoy the fullness of existence but find usefulness in the spaces between.

Thoughts weaken the mind. Craving withers the heart.

Success is as dangerous as failure. Hope is as hollow as fear.

The form of the formless. The image of the imageless. Elusive, it is beyond comprehension. Approach it, and there is no beginning. Follow it, and there is no end.

The Vibe Coder doesn’t seek fulfillment. Not seeking, not expecting, he is present for all that appears.

Empty yourself o...

A collection of principles and observations noted from the book, preserved for reference.

• Habit is second nature but an artificial habit is stronger than nature.

• To make money, requires a clear brain. A man has got to see that two and two make four; he must lay all his plans with reflection and forethought, and closely examine all the details and the ins and outs of business.

• The use of intoxicating drinks as a beverage, is as much an infatuation, as is the smoking of opium by the Chinese, and the former is quite as destructive to the success of the business man as the latter. It is an unmitigated evil, utterly indefensible in the light of philosophy; religion or good sense. It is the parent of nearly every other evil in our country.

Don’t Mistake Your Vocation

The safest plan, and the one most sure of success for the young man starting in life, is to select the vocation which is most congenial to his tastes. Parents and guardians are often quite too negligent in regard to this.

...

AI is the mirror to user's conciousness .

For good or evil AI will be used according to the wishes of its operator.

When asked what it was really liked to be a monarch, Queen Elizabeth || typically described it as a job. Several years ago a Russian Oligarch who had fled to England died from suicide because his wealth reduced from coul of billions to hundreds of millions !

In this series, I'll be sharing my own highlights from Eric Jorgenson's book "The Anthology of Balaji", which presents a curated collection of transcripts, tweets, and talks by Balaji. Content from the book is presented as-is, without any editing or interpretation on my part. My goal with this blog is to provide a useful future reference for myself and possibly for readers. I hope to create a valuable resource with this post.

MY HIGHLIGHTS

THE TYPES OF TRUTH

Most people do social diligence; Only a few do technical diligence.

Determining the type of evidence people accept is as important as knowing their incentives. Some take data , but many accept only popularity.

In theory, we could all just trust computer science. In practice, those who can't code will fallback on trusting in a computer scientist. Decentralizing truth-finding means enabling as many people as possible to do the math themselves.

SCIENTIFIC TRUTH

Only trust scientific truth what can be inde...

I recently set up DeepSeek Coder V2 using Ollama. It's a new AI model for coding and math tasks. The setup was simple, its just running following command : ollama run deepseek-coder-v2:latest

But I ran into a problem - the model was responding in Chinese instead of English.

Since the model is new, there weren't many guides available. After some searching, I found the issue on GitHub. The problem was in Ollama's template configuration. Ollama’s template had an extra space : Assistant: {{ .Response }}

Here's how to fix it:

1. Copy the model file: ollama show deepseek-coder-v2:latest --modelfile > new.modelfile
2. Edit new.modelfile. Replace the TEMPLATE string between double quotes with:

{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}User: {{ .Prompt }} {{ end }}Assistant:{{ .Response }}

3. Create a new model with the fixed file: ollama create deepseek-coder-v2-custom --file new.modelfile

4. Check if the new model is available : ollama list

After these ...

I think 100% decentralization is nearly impossible, and what we know as "decentralized" is a myth. It is definitely censorship-resistant by nature due to its architecture, but every network is vulnerable at some point.

What happened to the LBRY network? They announced their official closure in June 2023, citing a court loss and several million dollars in debts. Now, we might say that's because there was someone liable to answer to the SEC and the court, which isn't the case for the majority of blockchain networks. True, but they can seize domains, instruct exchanges to stop trading and facilitating tokens on their platform, and ask mining pools to stop supporting the mining of such tokens. Such actions may lead to a slow death for the network, like LBRY. A network is alive as long as users use the network, host nodes, mine, and trade tokens. If miners stop mining and people stop hosting nodes, the network will eventually die over time.

Every blockchain network node re...