The Pragmatic Engineer

AI tools for software engineers, but without the hype – with Simon Willison (co-creator of Django)The first episode of The Pragmatic Engineer Podcast is out. Expect similar episodes every other Wednesday. You can add the podcast in your favorite podcast player, and have future episodes downloaded automatically. Listen now on Apple, Spotify, and YouTube. Brought to you by: • Codeium: Join the 700K+ developers using the IT-approved AI-powered code assistant. • TLDR: Keep up with tech in 5 minutes — On the first episode of the Pragmatic Engineer Podcast, I am joined by Simon Willison. Simon is one of the best-known software engineers experimenting with LLMs to boost his own productivity: he’s been doing this for more than three years, blogging about it in the open. Simon is the creator of Datasette, an open-source tool for exploring and publishing data. He works full-time developing open-source tools for data journalism, centered on Datasette and SQLite. Previously, he was an engineering director at Eventbrite, joining through the acquisition of Lanyrd, a Y Combinator startup he co-founded in 2010. Simon is also a co-creator of the Django Web Framework. He has been blogging about web development since the early 2000s. In today’s conversation, we dive deep into the realm of Gen AI and talk about the following: • Simon’s initial experiments with LLMs and coding tools • Why fine-tuning is generally a waste of time—and when it’s not • RAG: an overview • Interacting with GPTs voice mode • Simon’s day-to-day LLM stack • Common misconceptions about LLMs and ethical gray areas • How Simon’s productivity has increased and his generally optimistic view on these tools • Tips, tricks, and hacks for interacting with GenAI tools • And more! I hope you enjoy this episode. — In this episode, we cover: (02:15) Welcome (05:28) Simon’s ‘scary’ experience with ChatGPT (10:58) Simon’s initial experiments with LLMs and coding tools (12:21) The languages that LLMs excel at (14:50) To start LLMs by understanding the theory, or by playing around? (16:35) Fine-tuning: what it is, and why it’s mostly a waste of time (18:03) Where fine-tuning works (18:31) RAG: an explanation (21:34) The expense of running testing on AI (23:15) Simon’s current AI stack (29:55) Common misconceptions about using LLM tools (30:09) Simon’s stack – continued (32:51) Learnings from running local models (33:56) The impact of Firebug and the introduction of open-source (39:42) How Simon’s productivity has increased using LLM tools (41:55) Why most people should limit themselves to 3-4 programming languages (45:18) Addressing ethical issues and resistance to using generative AI (49:11) Are LLMs are plateauing? Is AGI overhyped? (55:45) Coding vs. professional coding, looking ahead (57:27) The importance of systems thinking for software engineers (1:01:00) Simon’s advice for experienced engineers (1:06:29) Rapid-fire questions — Where to find Simon Willison: • X: https://x.com/simonw • LinkedIn: https://www.linkedin.com/in/simonwillison/ • Website: https://simonwillison.net/ • Mastodon: https://fedi.simonwillison.net/@simon — Referenced: • Simon’s LLM project: https://github.com/simonw/llm • Jeremy Howard’s Fast Ai: https://www.fast.ai/ • jq programming language: https://en.wikipedia.org/wiki/Jq_(programming_language) • Datasette: https://datasette.io/ • GPT Code Interpreter: https://platform.openai.com/docs/assistants/tools/code-interpreter • Open Ai Playground: https://platform.openai.com/playground/chat • Advent of Code: https://adventofcode.com/ • Rust programming language: https://www.rust-lang.org/ • Applied AI Software Engineering: RAG: https://newsletter.pragmaticengineer.com/p/rag • Claude: https://claude.ai/ • Claude 3.5 sonnet: https://www.anthropic.com/news/claude-3-5-sonnet • ChatGPT can now see, hear, and speak: https://openai.com/index/chatgpt-can-now-see-hear-and-speak/ • GitHub Copilot: https://github.com/features/copilot • What are Artifacts and how do I use them?: https://support.anthropic.com/en/articles/9487310-what-are-artifacts-and-how-do-i-use-them • Large Language Models on the command line: https://simonwillison.net/2024/Jun/17/cli-language-models/ • Llama: https://www.llama.com/ • MLC chat on the app store: https://apps.apple.com/us/app/mlc-chat/id6448482937 • Firebug: https://en.wikipedia.org/wiki/Firebug_(software)# • NPM: https://www.npmjs.com/ • Django: https://www.djangoproject.com/ • Sourceforge: https://sourceforge.net/ • CPAN: https://www.cpan.org/ • OOP: https://en.wikipedia.org/wiki/Object-oriented_programming • Prolog: https://en.wikipedia.org/wiki/Prolog • SML: https://en.wikipedia.org/wiki/Standard_ML • Stabile Diffusion: https://stability.ai/ • Chain of thought prompting: https://www.promptingguide.ai/techniques/cot • Cognition AI: https://www.cognition.ai/ • In the Race to Artificial General Intelligence, Where’s the Finish Line?: https://www.scientificamerican.com/article/what-does-artificial-general-intelligence-actually-mean/ • Black swan theory: https://en.wikipedia.org/wiki/Black_swan_theory • Copilot workspace: https://githubnext.com/projects/copilot-workspace • Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems: https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321 • Bluesky Global: https://www.blueskyglobal.org/ • The Atrocity Archives (Laundry Files #1): https://www.amazon.com/Atrocity-Archives-Laundry-Files/dp/0441013651 • Rivers of London: https://www.amazon.com/Rivers-London-Ben-Aaronovitch/dp/1625676158/ • Vanilla JavaScript: http://vanilla-js.com/ • jQuery: https://jquery.com/ • Fly.io: https://fly.io/ — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

1H 12min

Twisting the rules of building software: Bending Spoons (the team behind Evernote)Brought to you by: • The Enterprise Ready Conference on October 30th — For B2B leaders building enterprise SaaS. • DX — DX is an engineering intelligence platform designed by leading researchers. • ByteByteGo — Ace your next system design interview. — You may not be familiar with Bending Spoons, but I guarantee you’ve encountered some of their well-known products, like Evernote and Meetup. In today’s episode of The Pragmatic Engineer, we sit down with three key figures from the Italy-based startup: cofounder and CEO Luca Ferrari, CTO Francesco Mancone, and Evernote product lead Federico Simionato. Bending Spoons has been profitable from day one, and there's plenty we can learn from their unique culture, organizational structure, engineering processes, and hiring practices. In today’s conversation, we cover the following topics: • The controversial acquisitions approach of Bending Spoons • How Bending Spoons spent more than $1 billion in buying tech companies • How the Evernote acquisition happened • How Bending Spoons operates and how it organizes product and platform teams • Why engineering processes are different across different products • How ‘radical simplicity’ is baked into everything from engineering processes to pay structure. • And much more! — The Pragmatic Engineer deepdives relevant for this episode: • Good attrition, bad attrition for software engineers: https://newsletter.pragmaticengineer.com/p/attrition • Healthy oncall practices: https://newsletter.pragmaticengineer.com/p/healthy-oncall-practices • Shipping to production: https://newsletter.pragmaticengineer.com/p/shipping-to-production • QA across the tech industry: https://newsletter.pragmaticengineer.com/p/qa-across-tech — In this episode, we cover: (2:09) Welcome, Luca, Francesco, and Federico from Bending Spoons (03:15) An overview of the well-known apps and products owned by Bending Spoons (06:38) The elephant in the room: how Bending Spoons really acquires companies (09:46) Layoffs: Bending Spoons’ philosophy on this (14:10) Controversial principles (17:16) Revenue, team size, and products (19:35) How Bending Spoons runs AI products and allocates GPUs (23:05) History of the company (27:04) The Evernote acquisition (29:50) Modernizing Evernote’s infrastructure (32:44) “Radical simplicity” and why they try for zero on calls (36:13) More on changes made to the Evernote systems (41:13) How Bending Spoons prioritizes and ships fast (49:40) What’s new and what’s coming for Bending Spoons (51:08) Organizational structure at the company (54:07) Engineering practices (57:03) Testing approaches (58:53) Platform teams (1:01:52) Bending Spoons tech stack and popular frameworks (1:05:55) Why Bending Spoons hires new grads and less experienced engineers (1:08:09) The structure of careers and titles at Bending Spoons (1:09:50) Traits they look for when hiring (1:12:50) Why there aren’t many companies doing what Bending Spoons does — Where to find Luca Ferrari: • X: https://x.com/luke10ferrari • LinkedIn: https://www.linkedin.com/in/luca-ferrari-12418318 Where to find Francesco Mancone: • LinkedIn: https://www.linkedin.com/in/francesco-mancone Where to find Federico Simionato: • X: https://x.com/fedesimio • LinkedIn: https://www.linkedin.com/in/federicosimionato Where to find Gergely: • Newsletter: https://www.pragmaticengineer.com/ • YouTube: https://www.youtube.com/c/mrgergelyorosz • LinkedIn: https://www.linkedin.com/in/gergelyorosz/ • X: https://x.com/GergelyOrosz — References and Transcripts: See the transcript and other references from the episode at https://newsletter.pragmaticengineer.com/podcast — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

1H 18min

Episodes

Listen and read

Others have also enjoyed ...