Don’t mourn the web, bring it back to life
As a kid I learned python because I wanted to make videogames with the now long defunct Blender Game Engine.
This was between 2008 and 2012 and it was very fun. Allow me to reminisce for a while.
At the time, when I wasn’t at school, chances were I would be on the computer with at least one tab open on the Blender Brasil forum, dedicated to (Portuguese-speaking) people interested in computer graphics with free software. This was one of those PHP forums where you could add some markup with square brackets and emoticons and so on. We had about 5000 users, so that rarely 2 minutes passed without a new post or reply. But it was also a small enough place that a few mods could easily keep up with spam and misbehavior. There was a forum culture; when you got in, people acted a certain way, and you followed them.
People shared their cool 3D models, renders, animations, game prototypes and tutorials. They asked for help, artistic and technical. They formed little teams to do projects together. Most were beginners, many were literal kids (like myself). There was no “content”, people just did things.
We would chat on IRC. I made friends. I loved this; it was much easier to make friends on the internet than at school. We made silly joke IRC bots, little 2D and 3D games. I poked at Blender’s source code, others preferred using it. Some people really got better as time passed, and they were admired for the quality of their modelling, the grace of their animations, their lighting, their hand-drawn textures and material models, the photorealistic images they sometimes produced. I still remember quite a few usernames, but never met any of them.
Trying to learn computer graphics in that forum put me on a path to lots of creativity. I became fascinated with many topics and that would eventually lead to my carreer choices. I’m not saying I learned a bunch of stuff, or that it made me more intelligent or productive: I am saying I became aware of many things I would otherwise not know, tried my hand at them, dipped the toes of my mind in the Mysteries, and I was much better off for it. It doesn’t matter how competent I got in, say, programming through this exploration (in reality, not much). What matters is that I got myself out there and saw with my own eyes how big the world was.
Unfortunately, in today’s web there I do not see a lot of world travel. Well, there is travel, but it’s very little web hitchhiking, and very much web tourism. It’s incredible how streamlined the whole thing has gotten; as long, of course, as you wish “consuming content” to be the central goal. I am not the first to see this and lament it.
One can wonder how much of this sentiment is justified. After all, the world is still out there ready to be seen, even if we all flock to the same queues in the same ready-made instagrammable spots of the same cities. The existence of the tourism industry does not prohibit you in principle to just go cross your country on a motorcycle; the existence of Big Tech does not, in principle, prohibit you from going out there in the infinite hyperspace of the web and just be free.
At this point I instictively reacted with: “reality is not so simple. What good is being free if you are alone? There is no Blender Brasil forum anymore, and there hasn’t been one for years. Where did all those people go? Where would I go if I were a kid trying to learn to make cool videogames and animations nowadays?” This is what my “everything sucks now” response would be, with no data to back it up.
Then I started actually looking.
A naive Google search for “how to make 3D games” (query written in Portuguese, Google geolocalized in Germany because I didn’t bother finding a way around that) shows up YouTube™ tutorials and Reddit™ discussions. I’ll ignore Reddit because all I’ve seen there isn’t actually from Portuguese speakers; the discussions are auto-translated, sparse, and are mostly boring question-answer style. YouTube is interesting; I clicked on the first 10 videos or so and the spirit of those old forums is alive in the comments. A few of those tutorials will try to sell you courses, but this isn’t new, I remember paid course offers in the old days. In any case, these are just honest-to-God commercial offerings, shown to people who directly went looking for the thing that’s being sold, in a non-privacy-invading, non-evil-manipulative way. I’m totally fine with it.
People look fascinated in the YouTube comments. Wow, game engines? Wow, I can make my own 3D stuff that moves around? Wow, I don’t even need to know this galaxy-brain programming stuff to start? Wow, it’s free? Wow, I can do it directly in Android? “Hey I’m gonna remake Dark Souls with 7 monsters, rain, fog, each mob will have skills, Call of Duty guns, and Minecraft shields!!!”, says one commenter. “It’s my first game”, they add. Love the spirit.
I am not happy with the fact that these discussions are happening in YouTube comments rather than a free community space. But you know what? They are happening. Inside of this Skinner box hell designed for maximum “engagement”, curious and creative people do nonetheless talk to each other and figure stuff out.
YouTube comments are fine, but what about an actual forum? Well, kids these days call them Discord® Servers© (I like how corporate can just decide what a whole generation thinks the word “server” means. Reminds me of when Sky, the TV broadcasting company, tried to sue those who used the word “sky” for copyright infringement.) It’s not hard to find a Discord Server for Portuguese-speaking beginners learning to make videogames. I entered one, and it has around 1300 users with 300 currently online. The chats I could see without making an account don’t seem super active, there are 5-day gaps of activity. But the forum does exist, and people can find out about each other, share demos, talk about pixel art, sound design, game mechanic ideas and so on. I am sure the visitors of this walled garden are too busy having fun together to see the walls.
To make what could be a long post shorter: the web isn’t dead, so don’t mourn it. If you have the technical knowledge, make free community spaces, teach people to put up their own servers. Resist tracking, resist corporate control, preach against the attention economy, go all in. But if you can’t do this, don’t despair, don’t imagine that the bad ending is inevitable.
Just get out of the tourist traps and go see this foreign country for what it is.
Using fuzz testing in physics research
First, thanks to those who talked to me after last post. Your help is appreciated! I’ve received tips on using Julia’s environments and Manifest.toml to get things working on other machines with little fuss; I’ll definitely check that out, although for now what I want is to make a library, not a standalone program. It should be easy to call from Python and C, easy to vendor, and so on. I was also pointed to Julia’s discourse forum, I’ll ask there when I get the time.
Now properly speaking of fuzz testing / property-based testing, Supposition.jl, and how that’s helping me do research.
If you don’t know what those are, here’s a quick recap:
Writing unit tests for your code takes long, is boring, and mostly catches the bugs you anticipate anyway, so it’s not a great investment of your time. Fuzz testing means that instead you have the computer randomly generate thousands of tests, exploring many possible combinations of inputs that you couldn’t think of. Traditionally, a fuzzer will just check if your program crashes or not, but you can also ask other kinds of questions to it. You can ask things like: is the output of this function always a valid input to that function? Is FooError the only possible exception raised by this method? Is this equation always satisfied?
Asking those questions is checking properties of your code, and that’s where the name “property-based testing” comes from. In pratice, to do this, you have to
- Tell the computer how to generate inputs to your code
- Tell it what properties the code should satisfy
- Let it do its thing. It will find counterexamples and automatically shrink them to give you the smallest, simplest one that reproduces the bug.
Finally, if you write the tests first and the code later, this is called property-driven development (like TDD, test-driven development, but smarter). And Supposition.jl is a Julia package designed for property-based testing.
I started using Supposition.jl this Tuesday (4 days ago).
I’m having a great time with it! It didn’t take long to learn, the API is clean. And it has found so many useful counterexamples that when a test passes I’m really confident that the property in question is satisfied.
It has found trivial and shallow bugs, where I just didn’t understand Julia well enough (for example, type instabilities). It has found important, subtle bugs in my code where the logic was wrong, but required just the right combination of inputs to see. And it has found bugs in the math itself: I’m implementing stuff from a paper and it seems that some of its mathematical claims are wrong, because Supposition.jl found counterexamples.
This is great because I would never have thought of those counterexamples in so little time, or at all. It led me to a rabbit hole of math that was ultimately very productive, because now I have a much deeper understanding of what I’m doing, I know what the authors of the mistaken papers missed, how to fix it, and even how to make it more practical. Overall, great investment of my time as a researcher.
And to emphasize: I didn’t spend this week playing with Supposition.jl! I learned all I needed to learn from around half an hour in total looking up things in the documentation, and that was it. All my brainpower went to fixing bugs and understanding my math, while the fuzzer was the one trying to find clever counterexamples. I’m honestly impressed with how well it worked and how easy it was. It finds bugs faster than you can fix them.
It looks like fuzz testing is useful not just for software but for research. I’m definitely going to put any general claims I make in a future publication through a fuzzer before submitting it. It’s a cheaper, faster version of having your local mathematician check your stuff to find counterexamples; it’s not as rigorous, but damn does it catch edge cases.
A good physicist, like a good programmer, always takes the time to check that their reasoning is correct, at least in the common case. But it’s rare to have the time to rigorously verify that every step is completely correct in every possible case. Given that we are not going to use formal mathematics to prove our stuff correct anyway, fuzz testing is the cheapest, most effective option to find those mistakes. And I’m glad I’m using it.
I need help to make good scientific software
Part of my job as a PhD student is to write data analysis software for a future space mission. Because the launch is about 10 years into the future, there is still a lot of R&D being done, so all that I am writing now is really a prototype. It may or may not be used in the final pipeline.
Now, I really want my contribution to this effort to survive the next decade. I want it to be there, even if only in essence, in the pipeline that will run the first analysis of the mission data. But whether my code will make the cut depends on many things. Some of them could be grouped into the umbrella of “software quality”.
Software of good quality, in this context, is something that will happily work in other machines, years or decades from now, with no hassle. That won’t crash, won’t require a special environment to be set up for it, won’t yell at the user about dependency version incompatibilities. That will scale up and down to the size of the problem at hand and the computing resources available. Something that will just do its job efficiently, silently, and above all correctly.
In practice that’s just an ideal. We physicists are not very good at making robust, interoperable, pleasant to use software. We don’t have much training on software, we mostly learn to write for loops that compute big formulas. But I want to try.
The plea for help
Now, given that time is finite and my thesis won’t write itself, I need practical advice on doing this (if you have some experience and are interested, please give me a hand!). Some specific constraints are the following:
- I am pushing for the use of Julia in an environment dominated by Python and C, but I am not actually experienced in Julia. I just think it is a wiser engineering choice.
- I am writing signal processing and Monte Carlo routines.
- This has to run in standard Linux x86_64 clusters (slurm, kubernetes).
- My routines should be easy to call from Python and C for interoperability.
- We might want to throw GPUs at the problem.
What I thought so far
From my little experience I know that writing unit tests by hand takes time and can only catch the bugs you anticipate anyway; inline snapshot testing at least helps by making that easier. Random test generation seems promising and I will try out Supposition.jl very soon. Formal methods stuff like Alloy and TLA+ look fun but I have way too little time to learn them, and besides they seem most useful for concurrent algorithms which is not what I am doing.
On the robustness and ease of installation side of things, I want to avoid dependencies like the plague, unless they are very stable.
Fortunately for now I depend only on (other than Julia itself) the C library fftw and its Julia bindings. Those look stable enough to me that I shouldn’t worry about using them. I am wondering if I should vendor them to make things easy for the users and myself. Other Julia packages that I use (Revise, PyPlot) are only for development and so I don’t count them. Supposition.jl is an interesting case; if I do end up using it I should have some test suites that depend on it. That is still just a development dependency, but unlike Revise and PyPlot, its usage goes into the source tree. I have no idea what I should do then.
For interoperability with C, I know that compilation of Julia code into C libraries is possible but I never did it. The other direction should be easy.
For interoperability with Python, there is PythonCall which I hope won’t break too much. But if we are ambitious, in the long term this is not needed: all current Python code can be replaced by Julia. Not in the timeframe of my PhD, though.
Where that leaves us
This week I will try out this fancy property driven development thing. It looks like it can be very effective, and fun. Hopefully this won’t distract me too much from the actual thesis goals.
Anyway, cheers!
tags: software