I wanted to understand how @PyTorch works.

So I did the obvious thing:

I built it from scratch in C.

github.com/thevoxium/bare…

Some unknown time ago, humanity forgot to care about writing performant software. Nobody is looking at the instructions being sent to the CPU or the speed at which it can execute them. Your 100 GHz won't do shit beyond a certain point.

Somewhere, alone in a mountain bunker, there's still a dev working on hand-rolled assembly. Humanity's last hope.

static checking > dynamic checking but sir what you are doing is no checking at all.

“An ultraintelligent machine will be the last machine humans will ever make.” IJ Good

Humans should make intelligence abundant across the galaxy. As far as we know, we are the only intelligent civilization across the galaxy (I hope not). For over 13 billion years, the atoms in the universe have conspired together to create us. They created the conditions. They came together to give us intelligence. Now we should spread it everywhere. It should be our collective goal as a civilization. It is our duty.

There are a lot of rumors surrounding Recursive Self-Improvement (RSI). Obviously, these are just rumors for now, but for a moment, let’s treat it as a fact that some labs are actively trying to achieve RSI.

My first question would be: RSI in what domain? Do you even consider RSI to be domain-specific or generalized?

Let’s say we have a coding agent, A, and another agent, B. A is the base model, and B’s goal is to improve A’s benchmark scores on software engineering tasks. There are two possible ways B could achieve this.

One is that B somehow figures out how to directly modify A’s weights and repeatedly tests and evaluates the model’s outputs until it reaches its goal. The other is that B improves the “environment” used by A. This could include anything from the harness to the tools. It keeps doing this until it reaches its goal.

Which one of these is RSI? Let’s say both are.

Now, my question is this: If these systems use their knowledge and efforts to optimize the solution space for one specific domain, will they generalize to all domains? I think not. So, is this RSI? This cannot lead to AGI for sure, IMO.

There are other problems, too. For example, data. Synthetic datasets are used heavily in training these models. There is always a growing concern about model collapse, although I am not the best person to comment on that. I can, however, offer a simple thought experiment.

Let’s say LLMs produce verbose outputs, and these outputs become part of a synthetic instruction dataset. The post-training of new models will involve learning from these outputs. As a result, the new models will also prioritize producing more verbose outputs. If RSI relied on this methodology, there would be an explosion of verbosity, not intelligence.

Obviously, this could only happen if humans were no longer defining the systems around training.

But let’s say everything works out. RSI works. We now have a model that, at its current stage, is far more intelligent than the smartest human. That includes the ability to design the systems around its own training. Based on our premise, it can build much more efficient and intelligent systems than we can.

It will improve the systems, evaluate the results, update itself, and repeat the process. It sounds easy (it is not!), but let’s say it manages to do all of that.

And the model is not docile enough to tell us how to control it. What happens then?

I want to point out that this is a far-fetched idea right now. But IMO, if humans create a machine that initiates RSI and triggers an intelligence explosion, that should be the ultimate goal of humanity.

We cannot be like those puny alien civilizations hiding somewhere in the galaxy. Working toward an intelligence explosion should be our collective goal as a species.

We shall reach out to the stars. We will establish bases in other star systems. We shall make intelligence abundant across the galaxy.

A Lion Forgot How to Hunt

I went to a zoo for the first time in 15 years. I was just 10 the first time, and it was part of a school trip. I am an adult now. My memory does not serve me well, but I can say that my experiences on both occasions were completely different.

When I was a kid, all I remember is how amused I was. All-mighty animals, right in front of us, on display. This time, again, I appreciated the beauty of those animals. It was the first time I saw a tiger and a lion. Mighty animals. Mighty "beasts."

I felt some degree of empathy when I saw them. Why? I cannot clearly point to the exact reason, as there are many. But it is also strange that, in a jungle, they could rip me apart. Here, however, the most they could be were house cats. The roles were reversed. I was the king of the jungle this time, not that lion.

How must he have felt when I took his title? When WE took his title? Is he even aware that he is no longer the king of the jungle? Does that tiger know that I can no longer be hunted? Do they long for freedom?

I suppose that if there were a Socrates or Plato in the tiger kingdom, he or she might be thinking about ideas of freedom and open (dense) jungles, while the lions might be thinking about open savannas.

There were people calling out to them, teasing them, as if they posed no threat. They are now tamed, their freedom taken away from them. But they do get enough food. They do not have to hunt.

If I were a predator (which I am, I am a human, the best at it on this planet), and I got enough food every day without having to work for it, would I enjoy it? I do not know. I seriously cannot answer that as a lion. So perhaps the question itself was wrong. I cannot speak for that animal's feelings. That question is very wrong. Can a lion answer for me? Probably not.

See, basically, what I want an answer to is this: How do they feel now?

They are neutered. Their claws are no longer sharp. They have forgotten how to hunt. So, does hunting come naturally to predators, or is it simply a way to obtain food?

I was wondering what the human equivalent of this situation would be. What is it that we have forgotten?

Can I start from the beginning? Do I have your permission? I hope so.

Let us go back 10,000 years. We were hunters and gatherers. We do not do that anymore. I do not think 99% of us would survive if we were dropped into a jungle today. Those skills, passed down through our genetics, are long gone. The Industrial Revolution has diluted that connection. But obviously, we could quickly return to that way of life. Again, that is not the point.

Eventually, we became farmers. Again, that skill has been lost to most of the human population. We can learn it, but again, that is not the point.

What else? I do not recall us being anything other than this. Fighting in wars? We have always been experts in that field. So, that skill is not lost. Not YET.

Hmm, what else? The only thing I can think of is thinking, something that separates us from those mighty "beasts" (please keep focusing on those double quotation marks).

But again, while I am thinking, they might be thinking, too. Perhaps about hunting? I do not know.

How long until we lose our capability to "hunt"? How long until we become the mighty "beasts" enclosed in cages? How long until we have lost everything for comfort? For ease? How long until we cease to be human? How long until we are free?

How long until we become the lion that forgot how to hunt?

I would like to thank @cmuratori. I recently subscribed to his Computer Enhance Substack. The content on Performance Aware Programming, along with the effort that goes into producing it, is insane.

I am trying to watch at least one topic every day. Easily one of the highest ROI investments I have made recently to learn something new.

I hope this series continues for a long time and does not stop at Part 5.

Opus 5.5 made this with a single prompt using the OpenCode harness. I just asked it to make a simple video trailer for our Uzbekistan trip (which may or may not happen).

Obviously, it could have been much better, but I ran out of credits, and this took just 5 minutes, good eriment.

I don’t know anything about video editing. Zero clue. The best part of watching this happen was when it downloaded 30 different music samples and did a mathematical analysis of the sound levels to figure out which audio had the perfect levels for a crescendo and diminuendo. This was my first time making a video using tools with LLMs. But I would not even pay 1$ to a human if they gave me this, by that comparison, this is pretty shit.

Faster software is also greener software when you run it billions of times.

Do you'll think in this age of LLMs, where code is in abundance, the era of low level optimizations or working as close to hardware is over?

I have some crude thoughts about this. It is an interesting question. In my opinion, for most of the applications (not for web maybe!) the code was never the main, difficult job. It is true that it sits at the critical path of any application. So now that it is cheap, running code can still be considered as the bottleneck. We still have to worry about physics, you know, caches, bandwidths, and maybe even watts.

On the other hand, for most of the software, working close to hardware was maybe already over. Compilers got very good at local machine instruction generation. Hardware got much faster.

LLMs already output correct enough high level code. They can work on better algorithm selection, suggest data structures, or SIMD intrinsics, or if you have a benchmark in place they can iterate to improve the performance of hot loop. We have seen LLMs writing better cuda kernels than humans now. I have a contrasting opinion as well (Yes, as always, I remain conflicted in a lot of cases).

The percentage of bad code has gone up. These models are trained on the internet. Average code written by us developers is not machine friendly, it is allocation heavy, has a lot of cache misses, etc. You get my point. Data centers, clusters, all of them need software to be efficient otherwise billions of dollars will be wasted.

Maybe now we need to know why the CPU stalled, which transformation do we actually need, whether the optimization generated by the LLMs is real or not. There is now a infinite good looking code which may not function as well at closer inspection.

What are your thoughts about this?

Faster software is also greener software when you run it billions of times.

Do you'll think in this age of LLMs, where code is in abundance, the era of low level optimizations or working as close to hardware is over?

I have some crude thoughts about this. It is an interesting question. In my opinion, for most of the applications (not for web maybe!) the code was never the main, difficult job. It is true that it sits at the critical path of any application. So now that it is cheap, running code can still be considered as the bottleneck. We still have to worry about physics, you know, caches, bandwidths, and maybe even watts.

On the other hand, for most of the software, working close to hardware was maybe already over. Compilers got very good at local machine instruction generation. Hardware got much faster.

LLMs already output correct enough high level code. They can work on better algorithm selection, suggest data structures, or SIMD intrinsics, or if you have a benchmark in place they can iterate to improve the performance of hot loop. We have seen LLMs writing better cuda kernels than humans now. I have a contrasting opinion as well (Yes, as always, I remain conflicted in a lot of cases).

The percentage of bad code has gone up. These models are trained on the internet. Average code written by us developers is not machine friendly, it is allocation heavy, has a lot of cache misses, etc. You get my point. Data centers, clusters, all of them need software to be efficient otherwise billions of dollars will be wasted.

Maybe now we need to know why the CPU stalled, which transformation do we actually need, whether the optimization generated by the LLMs is real or not. There is now a infinite good looking code which may not function as well at closer inspection.

What are your thoughts about this?

I had to port a small internal agent-building platform that was previously built using LangGraph to now run on our own custom SDK. That SDK serves as a platform for building any type of agent internally. It handles auth, PII masking, retrieval, deep agents, custom workflows, and pretty much anything else we need. It was around a 2K LOC change. It took Claude 3 hours to finish everything. Every test case passed, and the system was deployed without any issues. This time, though, I took my sweet time to understand the architecture, since the platform was not built by me. For example, it was great to converse with Claude about why certain patterns were chosen, why things were designed a certain way, etc. I didn't write any code during this process, but I came out of it much more knowledgeable about the design choices that were made. There was a lot of back and forth with Claude. Also there was a funny situation where the model was stuck in a loop finding and killing ghost processes running on windows but was unable to do it. I watched primeagen's video yesterday and he talked about the whiteboard defense. And honestly, it made a lot of sense. In this rapidly changing world, the least I can do is understand the underlying design choices in detail. Alright see ya tomorrow. Debate with your model. Argue with it.

Now that the dust has settled and this post has been seen by almost everyone on tech Twitter, here is what I am gonna do next.

I want to make a community around this account, with people sharing their projects, ideas, insights, reposting, and quoting others, basically something niche. That will also inspire others seeing those posts to work on different ideas and learn new stuff. It is fine if you want to use LLMs, but it would be good if you keep sharing new learnings, how you are adapting, best practices, and so on.

Yes, we will have to adapt. Corporations will always optimize for higher profits, and if they think LLMs can drive that, then we will have to move in that direction. And it makes sense. As an employee, your only job is to generate profits. On the other hand, if you are the owner, you can decide the strategy you're gonna pick for the future. So, we need to get the paycheck, upskill, and try to keep your job.

Now what matters is what we do after work. I will start working on projects. It can be anything, using AI or not, but I will only rely on LLMs for boilerplate stuff. 1 hr daily will be more than enough. That's all it takes to keep the muscle memory intact. Learn the basics, read books, solve harder problems. I want to still code a lot by hand. I will do it in my free time and stop whining. I hope it makes sense.

So let's start building and learning new stuff every day. Post here. This community will motivate you.

Super long post ahead:

This resonated with a lot of people. We are all facing similar kinds of experiences in our work environments. Many suggested that I take it easy, just use LLMs to finish whatever task I have, get the paycheck, and be happy. I understand that sentiment. However, there comes a point in your life when you need to find meaning in your work.

It gets boring to do the same thing again and again. Your brain needs stimulation. You will start finding it somewhere if not in your work. But what if you do not have the time? How will you find your purpose then?

Quitting is not an option. You need that money. There are bills to be paid. So then, the option is to just hide your face in the dirt and do what is asked of you. Nobody will be there to listen to you. And why would they do that? The end goal of any business is to become profitable. And how will you do that? Ship faster, see what gets more users onboarded, retain them. Repeat.

There is no concept of soul in this process. There is no one who will care about your satisfaction. Just you. Find it somewhere.

Code in your free time.

You do not have free time.

Find it.

There is no other option. You want to feel something? Build it in your free time.

I agree with a lot of people here saying we will just be meat proxies. Tbh, I do not understand the term exactly, but I think I get the sentiment.

But on the other hand, what I worry about is that a lot of engineering domains will start lacking intellectuals.

Think about it. If you are a kid now, in school or college, all you need to do is ask LLMs to finish your homework. Teachers are aware of it, and there is nothing they can do. If I were a kid, LLMs would be my greatest friend. They would be my replacement for the hatred I had for assignments.

The art of learning will eventually disappear. Who will be there to teach, and who will be there to learn?

Eventually, we will have a generation of kids who have no drive and curiosity. But I am pretty sure that if we steer this technology in the right direction, the opposite could also be possible. I just do not see that happening anytime soon.

I digressed a bit, but what I am trying to say is that software engineers used to take a lot of pride in their work. There is still a possibility for that, but an MBA can now come to you and say, "Hey, I could have shipped that too. What have you done here?"

Are we ready to face that question?

I think not.

That is ego. That will eventually go away as we adapt. Some of us are already adapting. There is resistance, and there always will be. I think it is a natural progression during any technical revolution. Yes, it is a big leap in technology. We cannot deny it any longer.

What I am interested to know is how this will translate to other domains.

Software can be automated. Let us assume that.

What happens to math, physics, biology, medicine, mechanical engineering, aeronautics, and so on? Will they face a similar fate?

I do not have an answer to that.

Okay, now coming back to the topic. What do you think will happen to devs?

Since most of us, judging from the response to this post, are using LLMs to do whatever management asks, what will happen to us?

If models become super good, which I think they eventually will with the pace at which we are moving forward, will all of us eventually go out of a job?

I again do not have an answer to this.

Can someone focused solely on engineering survive?

Can someone focused solely on design survive?

What about someone focused just on PM?

Can an individual do the job of all three?

Maybe eventually they will have to.

So how do I see the future of software engineering now?

I think I need to reference the replies I have received on this post.

It is bleak.

At least the future of what we think of as traditional software engineering is bleak.

I am not even sure whether whatever comes next can be considered software engineering or not.

On a side note, I never even liked labeling building software as engineering. When I was a kid, I always associated engineering with making things like bridges, cars, rockets, etc.

So how will you find the hard problems to solve when you are not getting any time in your day apart from talking to Claude?

I do not know where we will find that.

Maybe I need to pick up a new domain.

Hardware?

Very interesting, but I am too dumb to learn about it now. And most probably, it is not going to pay my bills.

So not that.

And there is nothing else that comes to my mind apart from these two.

I think I am doomed.

I know I am just rambling in this post. Not a lot of it will make sense, but I need to just get it out.

Now, regarding this post, this is a once-in-a-lifetime reach I got on this one. I do not think this will happen again for a long time. Almost everyone on the tech side of X saw this and shared their opinions.

I am grateful for this.

This single post almost tripled my followers overnight. I am thankful to all of you for sharing your thoughts on this matter.

I would like you all to post more on X. Share your thoughts. It can be about anything.

Maybe this will give your brain the stimulation that coding at work is not going to.

Start today.

Post anything.

Try to be original.

I could just keep on writing. I like writing. I like writing long posts, even if nobody is going to read them.

Maybe that is what I like the most about X. Just sharing my thoughts and seeing what others say about them.

Interesting era that we all live in. I could say anything, and hundreds of people are going to see it.

Okay, what else?

Let us talk about life now.

I think, as I mentioned, the soul-sucking nature of AI is something that bothers me.

I would have loved AI to solve cancer.

It took my grandparents from me, and if it solves that, I do not think anybody would be as happy as me.

I need it to create more medicine, help humans live longer, and improve our quality of life.

Software automation is fine, but most of our world can live without our CRUD apps.

What it cannot live without is new medicine and cures for new diseases.

I hope AI gets so advanced that any disease can be cured using personalized medicine.

Not sure if I am making sense.

Anyways, this technology has had the most impact on my life. And this will continue to happen.

I hope that we humans are a part of this industry shift. I do not want to be sidelined. I want to cherish new technology.

Do you know how it would have felt to live during the space race?

I want to feel something similar.

The difference is that the space race was not threatening anyone's livelihood.

On the other hand, this time, all we hear from the AI industry is that they are going to make us jobless.

I hate that about them.

So negative.

It does not have to be like that.

I am tired of typing this now. I will share my thoughts sometime later.

Yes, I do have many thoughts.

That is all I do.

I think about anything.

At least no one is automating my consciousness yet.

I am done with this shit. It is over. The state of engineering right now is horrible. It has been half a month since I started a new role at a big company. Nobody knows anything here. The specs, code, tests, PRDs, tickets, resolution of those tickets, reports, etc., everything is made by Claude Code. Nobody on my team likes this. They are being forced to ship as much as they can. I have heard multiple times from higher management that pushing code is not a bottleneck, so why are we slow? People are working 12 to 13 hours a day just to press enter. Nobody is reading anything. Humans in corporate are doing nothing on their own. Everyone, literally everyone, from an L1 to an L7 engineer here is doing the same thing. Talk to Claude. There is no sense of victory. Nobody is resolving bugs. In reality, nobody is thinking anymore. Everything is done by LLMs. It is so soul-sucking. I would not mind it, to be honest, if we were at least given the time to check out the code and see what is going where. But no, the goal is to just ship. No matter what happens.

Whenever I see videos from American science and engineering YouTubers, honestly, the first emotion I feel is jealousy. I see their workshops, the tools they have access to, the sheer variety of them, all organized and readily available. The culture they have of building things on their own is, I think, one of the things that has made them great. They are the kind of people who figure things out.

If I had been a kid and my father had access to these tools, stores, parts, or even the desire to build things from scratch, I could have been something different.

The nature-versus-nurture debate doesn’t really seem to apply to America. Both seem to be conducive to sparking curiosity in kids. They have access to the best environment, nature included, education, and a strong DIY culture.

They have mall-sized hardware stores where you can buy almost anything you want. A specific type of nut? They have it. A particular type of spring? They have it. I have also noticed that they are collectors. They don’t necessarily care how expensive a piece of equipment is; if it has a use, they will get it.

Most of these things are not available to us.

A child born in India has to just focus on studies. Parents don’t want us to build things. For example, during summer vacations, all we have is a ton of homework to do, and much of it is just memorization. There is very little that sparks curiosity or ideas in a kid’s mind.

American kids go camping and hunting. They learn to do things on their own. Some of them may help their dads renovate their homes. I think some of this comes down to the dignity of labor. They don’t shame someone for being a plumber, electrician, or carpenter. These are respectable jobs, something I can’t say about my country.

I don’t know, man, what else to say. Just some random thoughts.

This here made me change my mind. Now I cannot say that these models are just probabilistic next-token predictors. Maybe those billions of neurons, trillions of parameters, working together, really do become AGI.

We are living in the "when decades happen in a week" era of innovation and scientific discovery.

Today was the day 2 at the new company, it is my first job switch. Onboarding is still going on, I will be posting less for the next week or so. Still understanding what is going on here.

Here's how to build an LLM inference engine from scratch:

The goal of an inference engine is to take a model and input and somehow efficiently produce the output tokens. Basically, we want to simulate a model's computation.

Think of it like this:
An LLM's computation flow starts from input tokens -> create embeddings -> pass through various transformer layers -> generate logits -> sample based on the probability -> autoregressive generation of the next token.

Our engine should execute all these ops efficiently. We need roughly 5 layers.

1. Model Representation
We need a way to load the trained model's weights along with its configuration. This can include:

- Embedding matrices
- Q, K, V matrices
- Feed-forward weights
- LayerNorm parameters
- Tokenizer configs
- Model config, like the number of layers, heads, d_model dimension, etc.

Start from an already-built format like safetensors and build something to load the weights. Do not build a new format in the beginning itself.

2. Core Tensor Operations
This can include matmul, vector add, element-wise multiplication, softmax, layernorm/rmsnorm, transpose, reshape, attention, some activation functions, embedding lookups, etc. Most of these are very simple ops and can be easily implemented in the language of your choosing.

Since we do not have to implement backprop for these ops, just implementing the forward pass is much easier.

3. Then we need to execute the transformer loop as described earlier. Your autoregressive loop should be working correctly at this stage, and now you have a beginning-level inference engine. It is that simple (make it first, improve it later situation).

4. KV Cache
A good inference engine should have a way to manage the KV cache to manage the O(N^2) complexity of the transformer (attention). The memory management should work well for the KV cache, weights, activations, temp tensors, and input/output buffers.

5. Now, once this basic engine is working on CPU, we need to make it fast.

For CPU, some things you can try include:

- SIMD
- Multithreading
- Cache-aware matmul
- Custom mem allocator
- Quantization

For GPU:
- CUDA/other DSL kernels
- cuBLAS for matmuls
- Kernel fusion (big improvement)
- Async execution

Now obviously this is all easier said than done, but we all need to start somewhere.