Software architect matklad advises researchers on mastering design through practice, not theory

A software architect known as matklad has responded to an inquiry from a researcher physicist regarding the acquisition of software design skills. Drawing on personal experience transitioning from a bioinformatics lab to leading major open-source initiatives, the author argues that software design is best learned through practical experience rather than formal courses. While acknowledging early exposure to design concepts at university, matklad characterised that theoretical education as largely make-believe, contrasting it with the lessons derived from real-world project leadership.

Central to the advice is the application of Conway's Law, which posits that software genesis repeats the social architecture of the organisation producing it. The author suggests that the perceived divide between industrial and scientific software is driven less by technical knowledge and more by the field of incentives compelling people to produce the code. In the context of scientific code, where tight publication deadlines often dictate timelines, matklad advises researchers to adapt to existing incentive constraints if they cannot be changed, rather than attempting immediate structural fixes.

To manage unideal incentive structures, the author proposes a strategy of "speedrunning the four stages of grief to acceptance". This approach acknowledges that incentive structures are rarely what one wants them to be and that projects must be done to the best of one's ability given the constraints. Matklad notes that this reality applies to most industrial software projects as well, where there is seldom a time to do a thing properly without first adapting to the environment.

Illustrating this point with the rust-analyzer project, matklad detailed how the build system was intentionally designed to balance deep technical work with broad community contributions. The project allowed "weekend warriors" to contribute features that might crash, while the core spine maintained higher quality standards. This segmentation was a deliberate choice to attract high-impact contributors without requiring them to build the entire compiler, ensuring that the bar for a feature pull request was that the happy path works and is tested.

Regarding concrete learning resources, the author explicitly stated that no single book contains the truths of software architecture, suggesting that practice is an essential element that cannot be fully captured in text. Nevertheless, matklad recommended several specific materials for further study, including Gary Bernhardt's "Boundaries talk" and Pieter Hintjens' "∅MQ guide". Additional suggestions included books such as "The Philosophy of Software Design" and "Software Engineering at Google", as well as reflective writings by Jamii and Ted Kaminski.

The author cautioned that adapting to, rather than fixing, an incentive structure carries uncertainty, as the future tends to happen in the least convenient manner. Despite the experimental nature of the code produced under such constraints, the goal remains to prototype better architectures that can be backported to core systems. Ultimately, the advice emphasises that an inquisitive mind can figure out software engineering from first principles, provided one engages with the social and practical realities of the work.