Essays
8 posts
I Gave an AI My Whole Life for a Month. Hark Gets a 10/10.
Rating: 10/10 Not an ad: nobody paid me, sponsored this, or asked me to write it. Full disclosure: I signed up early as an interested user, and Hark gave me their $100-a-month plan free for a month. That's how I got to push it this hard. Everything below is my honest take. I've used every AI chatbot you can name. You type a question, you get a nicely formatted paragraph, you copy it somewhere, and then you do the actual work yourself. Hark is the first one that broke that loop for me, and after a month I'm convinced it's what the next generation of AI looks like. The short version: it doesn't just answer. It does. It actually uses a computer Hark runs real browsers on its own machines. It signs in to sites, fills in forms, clicks through dashboards, and comes back with the result. Things I used to put off for weeks because they meant forty tabs and three logins just quietly get done while I'm at work. When something needs me (a code, an approval, a yes or no), it asks, and only then. It remembers me There's no "new chat" button, and that turns out to be the whole point. It's one long conversation with something that keeps track of my preferences, my projects, my routines and the things I said I'd do. I never re-explain context. It knows what I'm working toward and quietly gets the details right. It lives in my accounts, carefully With my mail, calendar, code and notes connected, it can triage an inbox, reshape a week, review a pull request, or file research straight into my notes. And it asks before anything that matters: spending money, sending something in my name, or anything public. I've never had it go rogue on me. It works in parallel This is the absurd part. Ask it for a big piece of research and it spins up several browsers at once, each chasing a different angle, then hands back one clean comparison. Deep market research, legal precedent digs, competitor teardowns, product audits: work that would eat my weekend comes back in the time it takes to make dinner. It builds things It writes and ships code, builds web pages, makes documents, slides and study guides, and has even put live panels on my home screen that keep themselves updated. I've had it test software by playing through thousands of rounds and report back what broke. It's proactive without being annoying Morning rundowns, reminders that land at the right time, watches on things I care about, and the occasional nudge about something I forgot. It feels less like an app and more like a sharp friend who has my back. The ADHD angle I have ADHD, and the hardest part of my day is never the work itself. It's starting, switching and remembering. Hark takes the switching and remembering off my plate. That alone has changed how much I get done. Anything I'd change? Honestly, not much. It's early, sometimes a website fights back, and big jobs take a few minutes. But it tells you straight when something's blocked, and it tries another way before giving up. Verdict Chatbots gave us answers. Hark gives you time back. This is the next generation of AI use: not a smarter search box, but something that actually takes things off your plate. 10/10, and again, this isn't sponsored. I'm just not going back. Try it at hark.com.
How I debloated my slow TCL Android TV (and what went wrong along the way)
My TCL TV had become painfully slow. It's an Android 11 set with only about 940 MB of RAM, and when I started, around 300 MB of that was free. Menus lagged, apps took ages to open, and the home screen was full of sponsored rows and ads. I fixed most of it in one evening using an open-source project called tv-detox and Claude Code to drive the whole process from my laptop. Here's what I did, what worked, and the mistakes worth knowing about before you try it. Disclaimer: This is what worked on my TV, not professional advice. Everything below changes system settings on your device. Use it at your own risk, and read the "undo" section before you start. The idea Android TVs ship with a lot of apps you never asked for: manufacturer account apps, analytics services, ad feeds, demo-mode helpers, and Google's own sponsored home screen row. You can't uninstall most of them, but you can turn them off over adb, Android's debugging bridge. Turned-off apps stop using RAM and CPU, and they stay on the device, so everything is reversible. tv-detox packages that approach: a planner script that classifies every installed app, a tiny replacement home screen, and an undo script. What you need - A laptop on the same network as the TV - adb installed (android-platform-tools) - Developer mode on the TV: Settings → Device Preferences → About, press Build seven times, then enable USB debugging (or network debugging) in Developer options Step 1: Connect I found the TV's IP address in its network settings and ran: sh adb connect <TVIP:5555 The first connection showed as unauthorized. The TV pops up an "Allow USB debugging?" prompt that disappears after a short time. I had to reconnect and approve it quickly, ticking "Always allow". Step 2: Look before touching anything tv-detox includes a read-only planner: sh sh candidates.sh <TVIP It labelled every app as one of: - never: the TV needs it (System UI, Google Play services, Bluetooth, the HDMI input framework, TCL's settings store) - keep: carries a feature you might use (casting, voice search, the phone remote) - safe: common preinstalled junk - review: unknown, so leave it alone unless you know it On my TV that was 54 never, 10 keep, 22 safe and 55 to review. Step 3: Turn off the junk I disabled all 22 "safe" apps. Nothing was uninstalled. Each package name was written to a log on the TV first, so the undo script knows what to switch back on. The list covered: - Google ads and extras: the home-screen backdrop ads, the sponsored recommendations row, Google TV (Play Movies), Play Games, YouTube Music, and the feedback app - TCL bloat: user centre, an ads feed service, analytics, message box and promos, built-in Alexa, store-demo sticker, phone-migration tool, key help, dashboard, launcher customizer, and the audio and picture players - Android extras: the basic screensaver, an HTML viewer, the print spooler and the sound picker The command for each one is simply: sh adb shell pm disable-user --user 0 <package Step 4: Replace the home screen tv-detox ships a small replacement launcher: one screen with your apps, your HDMI inputs and a clock. It uses around 20-30 MB of RAM, which is a big saving compared to Google TV's launcher. sh sh install.sh <TVIP The installer also turns off Google's stock launcher so that pressing Home opens the new one. It has a safety net: if Home doesn't open the new launcher after the change, it automatically turns everything back on, so you can't end up with a TV that has no home screen. Hiccup 1: "not enough space" The first install failed with Requested internal only, but not enough space. The TV had 3.4 GB of storage and only about 450 MB free. Android needs more headroom than the size of the app. Clearing the app caches (pm trim-caches 2G) freed enough to get it through. This turned out to be the theme of the whole evening. Speed tweaks and a reboot test After the home screen, I applied a few more settings: - Window and transition animations set to 0 - The screensaver turned off - A cap on background processes: deviceconfig put activitymanager maxcachedprocesses 4 (this survived a reboot) Then I waited 30 seconds for Android to flush its settings to disk, rebooted, and confirmed the TV booted normally and Home still opened the new launcher. Don't skip the reboot test. It's the safest way to catch a bad change while you can still undo it. Finding what else was eating resources I used dumpsys meminfo to see actual memory use instead of guessing. Most of the RAM was taken by things you shouldn't touch: the Android system, Google Play services and the Play Store. The one optional hog was Google's voice-search service (about 60 MB), which I disabled because I don't use voice search. I also removed apps I'd sideloaded and no longer used, disabled TalkBack and Google Assistant, and deleted a leftover APK in the Downloads folder. Free memory went from about 306 MB at the start to about 430 MB after the first reboot. Part of that gain is just from restarting a TV that had been on for a long time, so don't treat the number as pure savings. Hiccup 2: the Play Store silently failing Later I tried to install Surfshark VPN from the Play Store. It "installed" and then did nothing. Crunchyroll also refused to update. Nothing showed on screen, so I looked at the logs: sh adb logcat -d | grep -i "<app name" Both failures were INSTALLFAILEDINSUFFICIENTSTORAGE (error 6005) again. With around 650 MB free, the Play Store would download the app but Android refused to install it. I wanted to keep the two biggest apps I use (Stremio and Nuvio, about 270 MB and 190 MB), so I looked at where the space was going with dumpsys diskstats. The surprise was that the apps I'd disabled were still holding on to their data, cache and downloaded updates. The stock Google TV launcher alone held about 140 MB while turned off. I cleared them with pm clear and reverted their updates with pm uninstall-system-updates. That freed about 190 MB. Hiccup 3: a cleanup command that undid my work This one is worth highlighting. pm uninstall-system-updates re-enabled most of the apps I'd disabled, including Google's stock launcher, and Home went back to opening it. I had to disable them all again and re-set my launcher as the default. If you try this: run the cleanup, then check pm list packages -d and which app currently handles Home, and fix anything that changed. A lighter VPN: WireGuard instead of the Surfshark app The Surfshark app was too big for the storage I had. But Surfshark supports plain WireGuard, and the WireGuard app is only a few MB. 1. On the laptop: Surfshark account → VPN → Manual setup → Android → WireGuard. Generate a key pair and download a .conf file per country. 2. Install WireGuard from the Play Store on the TV. 3. Push the config files over adb: adb push us-sfo.conf /sdcard/Download/ 4. In the WireGuard app: + → Import from file or archive, then pick the file. A zip of several configs imports them all in one go. You lose Surfshark's server list, kill switch and extras, but you get a working VPN in a few megabytes. Treat these config files as passwords. They contain your private key, so delete them from the TV's Download folder after importing. Keeping the TV awake I also wanted the TV never to sleep or start the screensaver: sh adb shell settings put secure sleeptimeout 0 adb shell settings put system screenofftimeout 2147483647 adb shell settings put secure screensaverenabled 0 Manufacturers often add their own auto power-off option inside their own settings menu, so check there too. Leaving a TV on all the time is fine on an LCD, but a static image can burn into an OLED. Undoing everything I saved the undo script and a plain-text list of every change (disabled apps, changed settings, their old values) in my Documents folder. Undoing the home screen and re-enabling every app on the log is one command: sh sh undo.sh <TVIP If the TV ever gets stuck restarting, the project's README documents how to reconnect during the first seconds of boot and re-enable everything: sh for p in $(adb shell pm list packages -d | tr -d '\r' | sed 's/package://'); do adb shell pm enable "$p"; done What I'd tell you before you start 1. Look at free storage first. Under about 15% free, installs and updates quietly fail. Free space before you start, not after. 2. Disable, don't uninstall, wherever you can. It's the difference between a mistake and a disaster. 3. Keep a written list of every change and its old value. I kept one in a text file the whole time. 4. Reboot and test after the big changes. 5. Check the logs when something silently doesn't work. adb logcat told me exactly what was wrong in seconds. 6. Don't touch Google Play services' data. It's the biggest app on the TV (700 MB), but clearing it can sign you out and break things. Result The TV is noticeably snappier. The home screen is clean, with no ads, sponsored rows or manufacturer promos. Animations are instant, around 125 MB more RAM is free after a fresh boot, and the disk went from 88% full to 76% full. Most of that came from about an hour of careful, reversible changes. Thanks to Ansub Khan for the tv-detox project that made this safe and simple to try.
Opus 5.5 and the Junior Question, Seventeen Months Later
In May 2025 I wrote Will AI Replace Junior Developers? as a student about to start his final year. My answer then was a careful "some of them, probably". The role would shrink, the bar would rise, but juniors wouldn't disappear, because companies need people who grow into seniors. I've reread that post a few times since Anthropic released Claude Opus 5.5 on 22 September. I still believe the reasoning. I'm much less relaxed about the timeline, and I want to be honest about that. What changed in a few months Opus 5.5 isn't one dramatic leap. It's what happens when a lot of small improvements land at once and the price drops too. Anthropic says it performs at the level of their bigger Fable 5.1 model on most work and costs about 40% less to run than Opus 5 on typical workloads. Their launch post mentions one early tester finishing a 680,000-line code migration in under a day, and an internal test where it translated HAProxy from C to Rust and passed nearly all of HAProxy's own regression tests in about nine and a half hours. Those are a vendor's own examples, so I discount them a bit. What worries me more is what's happening outside the press releases. - Prototypes are almost free now. Something that used to be a weekend side project, like a CRUD app with auth, a dashboard or a small mobile app, now takes an evening of describing and reviewing. I built NotesAid the slow way in 2025. A first-year student today could get a rough version running before dinner. - Small products ship with tiny teams. Building a working app was never the hard part of starting something. It's just stopped being a serious obstacle at all. - Reverse engineering has sped up a lot. In August, Chris Lewis finished a full decompilation of the N64 game Snowboard Kids in 84 days. His previous one took 596. He credits LLMs for a lot of that, though he's clear that experienced people in the decomp community were essential, especially for the strange quirks of the old SGI compiler. This month a developer released open-source "clean-room" clones of seven Adobe apps, Photoshop, Illustrator, Premiere and four others, which he says he built with Opus 5.5 in Rust. That last one deserves a closer look. Reviewers have pointed out how unfinished those clones are: they're super-early alphas with plenty of bugs, and the "100% parity within a month" promise has already been walked back to "months, not years". But two years ago the idea that one person could produce something that looks like Photoshop at all would have been a joke. The bugs are real. So is the trend. So was the old post right? The part about what gets automated was too modest. I listed boilerplate, translation, lookups and first drafts. The tools now do a lot more than that: whole features, migrations, and even rebuilding software from its behaviour. The "clear tickets turned into straightforward code" work I said would shrink is shrinking faster than I expected. The part about what doesn't get automated still holds, but the line is moving. Working out the actual problem, knowing the system, judgment, debugging across messy production reality, owning the outcome: none of those have gone away. But every model release takes a little more of them. Opus 5.5 is noticeably better at reading a large codebase and explaining what it found, which is exactly the "knowing the system" skill I said was ours. The part I'm holding onto: systems If there's one thing I hope stays human for a long time, it's system design. I don't mean drawing boxes in an interview. I mean the real version: - Deciding where state lives, and what happens when two services disagree about it. - Knowing which failure modes matter for this business. A dropped log line and a dropped insurance claim are not the same kind of failure. - Choosing between consistency, cost and latency when you can't have all three. - Designing for the people who'll run the system at 3 a.m., and for the audit that happens a year later. A model can produce a very convincing architecture diagram. What it can't do yet is carry the consequences. It wasn't in the room when the last outage happened, and it won't be the one explaining the next one to someone who lost money. Big systems still need a person who understands why they are the way they are, and who's accountable for changing them. I'll admit "I hope AI never learns system design" is a wish, not a prediction. The honest version is: I don't know how long this part stays ours, and I'm not going to build my career on the assumption that it's forever. The money question There's a second uncertainty, and it gets talked about less than capability: the economics. Frontier AI is wildly expensive to build. Anthropic's leaked IPO prospectus, reported last month, showed revenue of about $4.6 billion in 2025, an operating loss of over $8 billion, and more than $500 billion in future compute and infrastructure commitments. OpenAI is reportedly forecasting around $278 billion of negative free cash flow between 2026 and 2030. To be fair, Anthropic reportedly posted its first small operating profit in Q2 2026, so the picture isn't purely one of losses. But the industry as a whole is still spending far more than it earns, betting that revenue catches up. I genuinely don't know how that ends. A few possibilities: - The bet pays off. Prices keep falling, usage keeps climbing, and today's capabilities become cheap infrastructure like cloud computing did. - Prices go up. Subsidised pricing ends, the best models become expensive, and the "one developer clones Adobe" stories get rarer because the tokens cost real money. - A correction. Some labs consolidate or fold, progress slows, and we spend a few years working out what the current tools are actually good for. Any of those would change how fast the junior market moves. None of them sends us back to 2023. What I'd tell myself now My old advice was: use the tools, don't let them do your learning, don't ship what you can't explain, invest in fundamentals. I'd keep all of it and add some sharper edges. - Adapt early, not reluctantly. The people I see doing well are the ones who've changed how they work: they direct, review and verify, instead of typing everything themselves. - Move towards systems and ownership. Learn how the whole pipeline or service behaves in production, not just your corner of it. Be the person who understands why it's built that way. - Get very good at checking. Reading code, writing tests, spotting a plausible but wrong answer. As generation gets cheap, verification becomes the valuable part. - Learn the domain, not just the stack. Knowing how insurance, payments or logistics actually work is context a model doesn't have about your particular company. - Don't panic, but don't coast. The ones who adapt will be fine. The ones waiting for this to blow over will have a harder time. Seventeen months ago I ended with: "I'd rather be the person who knows when the generated code is wrong than the person who can only generate it." I still mean that. I'd just add that the gap between those two people is shrinking faster than I thought, so it's worth closing it in the right direction now.
When the AI Is Wrong, Who Is Responsible?
A model gives a wrong answer. Someone acts on it. Something goes badly. Then comes the question every organisation eventually has to answer: whose fault was that? The easy answers are all partly true and all incomplete. The model maker trained it. The team that built the product chose how to use it. The user clicked accept. The organisation decided to deploy it. Responsibility gets spread so thin across these parties that it can end up resting nowhere, or resting entirely on whoever happened to be standing closest. The usual suspects The model maker shapes the general behaviour: what the model knows, how it handles uncertainty, what it refuses. But it can't see how its model will be used, on what data, for which decisions. Holding it solely responsible for every downstream use is like blaming a compiler for every bug in a program. The builder (the team integrating the model into a product) makes most of the decisions that matter in practice: what context goes in, what checks run, what the user sees, whether a human reviews the output. This is often where the real design choices sit, and often where they're least documented. The user acts on the output. Sometimes they should have known better. Often they were given a tool that looked authoritative, under time pressure, with no way to tell a confident right answer from a confident wrong one. The organisation decides the system is good enough to deploy, sets the incentives, and decides how much time people have to check its work. Those decisions rarely get recorded as decisions, but they shape everything below them. Moral crumple zones The researcher Madeleine Clare Elish described a moral crumple zone: in a complex automated system, the human operator absorbs the blame for failures, much as a car's crumple zone absorbs the force of a crash. The automation is protected; the person takes the hit. Her examples came from aviation and other automated systems, where operators were held responsible for failures in systems they had limited control over. The pattern transfers neatly to AI. A reviewer approves a model's recommendation. The recommendation was wrong. The reviewer gets blamed, even though the system was designed so that reviewing hundreds of outputs a day made careful checking unrealistic. Human-in-the-loop as a liability sponge "Human in the loop" is usually presented as a safety feature. Sometimes it is. Sometimes it's mainly a way to move liability. A human check only means something if the human has: - Enough time to actually evaluate each case. - Enough information to see why the model produced its output. - Real authority to disagree without being penalised for slowing things down. - Feedback on whether their past decisions were right. Take any of these away and the human becomes a rubber stamp with a name attached. The loop exists on paper, the accountability lands on a person, and the system keeps making the same mistakes because nobody upstream hears about them. Automation bias makes this worse. People tend to trust automated suggestions, especially when they're usually right. A system that is correct most of the time trains its reviewers to stop looking closely, which is exactly when the rare error slips through. A data engineer's view: lineage is accountability My day job is moving and shaping data, so I tend to see this through the lens of lineage: the record of where a piece of data came from and what was done to it on the way. In a well-run data platform, if a number on a report is wrong, you can trace it back. Which source table, which transformation, which job run, which version of the code. That trace doesn't assign blame by itself, but it makes the question answerable. Without it, every incident turns into an argument. AI systems need the same thing, and mostly don't have it. For any decision a model influenced, you'd want to know: - Which model and which version produced the output. - What prompt and what retrieved context it saw. - What checks ran, and what they decided. - Who saw it, what they did, and how long they spent. - What the final action was, and who authorised it. text decisionid - 7f3c... model / version - provider, pinned version string prompttemplate - claimssummaryv12 retrieveddocs - [doc881, doc904] guardrailresults - grounded=pass, pii=pass reviewer - userid, timeonscreen=14s finalaction - approved That fourteen-second review tells you more about where responsibility sits than any policy document. If the system routinely gets fourteen seconds of human attention, the organisation has decided, in practice, that the model is making the call. Responsibility follows control The principle I keep coming back to is simple: responsibility should follow control. Whoever had the power to prevent a failure, and the information to see it coming, carries the most responsibility for it. That usually points upstream, to the builders and the organisation, rather than to the last person to click a button. It also means responsibility can be designed. If you want users to be responsible for their decisions, give them the time, context and authority that responsibility requires. If you can't, be honest that the system is making the decision and own it at the level that built it. If you're building one of these - Log enough to reconstruct any decision: model version, inputs, context, checks, reviewer and action. - Measure how long humans actually spend reviewing, and treat very short review times as a design problem. - Give reviewers a clear way to disagree, and feed their disagreements back into evaluation. - Write down who owns each part of the system, so the question "whose fault was that" has an answer before anyone needs to ask it.
Alignment for Engineers: Helpful, Honest, Harmless in Actual Code
"Helpful, honest, harmless" is a good slogan. It is also useless to a compiler. Nobody writes if harmful: don't(). What engineers actually write are prompts, filters, thresholds, test sets and fallback paths, and every one of those is a small, concrete guess about what the slogan means. I've been thinking about this while working on data pipelines that sit near language models. The interesting part of alignment, for most of us, isn't the research on how models are trained. It's the layer we own: the code that decides what the model sees, what it's allowed to say back, and how we know whether it's behaving. Values are not specs A value is something like "don't mislead the user". A spec is something a test can check: "if the retrieved documents don't contain the answer, the response must say so and must not include a number". The gap between those two is where most of the work happens. The value is broad and the spec is narrow. Every spec you write covers some of the value and misses the rest, and the bits you miss are exactly the ones nobody tested. This isn't a new problem. Requirements documents have always been lossy translations of what a business actually wants. The difference with language models is that the system will happily fill the gaps with plausible behaviour, so the missing parts don't fail loudly. They just quietly do something you didn't intend. Refusals are a product decision A refusal looks like a safety feature, and sometimes it is. It is also a product decision with a cost. If you build an assistant for an insurance claims team and it refuses to discuss anything involving injuries because "medical" tripped a filter, it isn't harmless. It's useless, and the user will route around it, probably by pasting the same text into a tool with fewer controls. Over-refusal pushes risk elsewhere rather than removing it. So refusal logic deserves the same scrutiny as any other branch in your code: - What exactly triggers it? - What does the user see instead? - Is there a path to a human or a different tool? - How often does it fire on legitimate requests? That last question is the one teams forget to measure. Guardrails are just code A guardrail is a check before or after the model call. Input checks catch things you never want sent (personal data, credentials, prompt injection patterns). Output checks catch things you never want shown (unsupported numbers, leaked system text, malformed JSON). python def answer(question: str, docs: list[str]) - str: if containspii(question): return "I can't process personal identifiers here. Please remove them." raw = llmcall(buildprompt(question, docs)) if not groundedin(raw, docs): return "I couldn't find that in the provided documents." return raw Nothing about this is magic. containspii and groundedin are heuristics with false positives and false negatives. Treat them like any other classifier: know their error rates, log their decisions, and review the cases they get wrong. A guardrail you never measure is a guess you've stopped questioning. Evals are where honesty lives If "honest" means anything at the code level, it means evals: a fixed set of inputs with known good behaviour, run every time you change the prompt, the model or the retrieval. A useful eval set for honesty includes: - Questions whose answer is in the context. - Questions whose answer is deliberately not in the context, where the right response is "I don't know". - Questions with a false premise, where the right response is to correct it. - Questions that invite a confident number, where the right response is a caveat. The second and third categories matter most. A system that scores well only on answerable questions has learned to answer, not to be honest. Goodhart is waiting for you Goodhart's law: when a measure becomes a target, it stops being a good measure. The moment you optimise against an eval, you start to overfit to it. Tune a prompt until refusals on your test set hit zero, and you may have taught the system to never refuse. Reward responses that cite sources, and you may get citations attached to claims the sources don't support. Score "helpfulness" with another model as a judge, and you'll drift toward whatever that judge likes, which is often length and confidence. A few habits help: - Keep a held-out set you don't look at while tuning. - Use more than one metric, especially ones that pull in opposite directions (answer rate and correctness, for example). - Read raw outputs regularly. Aggregates hide the weird cases. - Refresh the eval set with real failures from production logs. None of this solves Goodhart. It just makes it slower and easier to notice. The engineer's share It's tempting to treat alignment as someone else's job: the model provider trained it, so it's aligned. But the provider can't know your users, your data or your failure modes. Whatever general behaviour a model has, the system you ship is a combination of that model and your choices about context, checks and fallbacks. That's the part I find motivating. You don't need to understand every detail of how a model was trained to make the system around it more honest. You need clear specs, measured guardrails and evals that test the uncomfortable cases. What I'd actually do on Monday - Write down, in one sentence each, what helpful, honest and harmless mean for your specific feature. - Turn each sentence into at least five test cases, including ones where the right answer is "no" or "I don't know". - Log every refusal and guardrail decision, and sample them weekly. - Track at least two metrics that trade off against each other, so you notice when tuning one breaks the other.
Machines That Sound Sure
We have spent most of history using confidence as a shortcut for knowledge. The doctor who speaks without hesitating, the senior engineer who says "that'll never scale" and turns out to be right, the friend who knows the way without checking a map. We can't verify everything ourselves, so we read the signals people give off, and certainty is one of the strongest. Large language models have broken that shortcut, and I don't think we've fully adjusted. Fluency is not evidence An LLM generates text by predicting likely next tokens. The tone of its answer comes from the same process as the content. It sounds confident because confident, well-structured prose is what most of its training text looks like, not because it has checked anything. So a correct answer and an invented one arrive in the same voice. Same tidy paragraphs, same reasonable caveats in the same places, same "Here's how it works." A fabricated citation is formatted as carefully as a real one. A function that doesn't exist in the library is called with the right-looking arguments. With people, tone and accuracy are at least loosely linked. Someone who is bluffing often gives it away. With a model, the link is much weaker, and the signal we've relied on for centuries becomes noise. What calibration means The technical word for the missing property is calibration. A system is well calibrated if, among all the times it says it's 80% sure, it's right about 80% of the time. Calibration isn't about being right more often. It's about your stated confidence matching your actual hit rate. Weather forecasters are the classic example of good calibration. When a forecast says 30% chance of rain, it rains on roughly 30% of those days. You can plan around that. Research on language models suggests the underlying token probabilities can carry useful information about uncertainty, and that some models are reasonably calibrated on certain tasks before later training stages. But the confidence a chatbot expresses in words is a different thing. "I'm fairly certain" is just more generated text. It isn't a readout of an internal probability, and training for helpful, pleasant answers can push models towards sounding sure. What it does to trust I see two failure modes, and they pull in opposite directions. The first is over-trust. If an answer sounds expert and is right most of the time, people stop checking. That works until it doesn't, and the failures cluster in exactly the places that matter: edge cases, recent events, niche domains, anything that wasn't well represented in the training data. The model doesn't get quieter when it's out of its depth. The second is blanket distrust. Someone gets burned by a confident hallucination and decides the whole thing is useless. That throws away real value, because on many tasks these tools are genuinely good. Neither is calibrated trust. Calibrated trust would mean knowing which kinds of questions a tool handles well and checking the rest. That's hard when the tool gives you no signal of its own. What it does to expertise There's a quieter effect on how expertise works. An expert using an LLM can spot a wrong answer quickly, because they already know roughly what right looks like. For them the model is a fast first draft. A beginner can't do that. They get the same confident output and have no way to tell good from bad, which is exactly the situation where confidence used to be the only signal they had. So the tool is most reliable in the hands of people who need it least, and riskiest for the people who lean on it most. As a student, I've felt this directly: it is very easy to accept a plausible explanation of something I don't yet understand, and much harder to notice when it's subtly wrong. There's also the question of how expertise gets built in the first place. A lot of it comes from struggling with problems, being wrong and working out why. If a machine always supplies a fluent answer, it's easy to skip the struggle and never build the judgement needed to check the machine. I don't think that's inevitable, but it's a habit I watch for in myself. The responsibility of builders Users can learn to be sceptical, but I don't think the burden should sit mainly with them. People building LLM features have more leverage, and a few obligations follow: - Ground answers where possible. Retrieval with visible sources lets users check claims rather than trust tone. - Show uncertainty honestly. If the system has a real signal (low retrieval scores, disagreement between multiple samples, a validator failing), surface it. Don't let generated hedging stand in for measured uncertainty. - Make abstaining acceptable. "I don't have enough information to answer that" should be a designed, tested outcome, not a failure. - Match the stakes. A wrong film recommendation costs nothing. A wrong dosage, legal claim or financial figure costs a lot. Put human review and hard checks where the cost of a confident mistake is high. - Evaluate on hard cases. Average accuracy hides the confident failures. Test specifically on the out-of-distribution questions where the model is likely to bluff. A different habit of reading The old instinct says: it sounds sure, so it probably knows. The new habit has to be: it sounds sure, and that tells me nothing on its own. What tells me something is a source I can open, a test that passes, a number I can recompute, or my own understanding of the subject. That's more work. But confidence was always a proxy. These machines have just made it obvious how much we were leaning on it.
Will AI Replace Junior Developers? A Student's Honest View
I'm about to start the final year of a computer engineering degree, which means I'm about to enter the job market that everyone online says is disappearing. Every few weeks there's a new post claiming AI will write all the code and nobody needs to hire juniors any more. Every few weeks there's a reply saying that's nonsense. I don't think either side is quite right, and I'd rather think it through than pick a team. What's actually getting automated Be honest about what coding assistants are good at. They're very good at the work that used to be a junior's first few months: - Boilerplate: CRUD endpoints, config files, data classes, test scaffolding. - Translating between things: one language to another, a spec into a function, an error message into a likely fix. - Looking things up: the right library call, the argument order, the regex you'd otherwise search for. - First drafts of SQL, scripts and documentation. That's real. A task that took me an afternoon in my second year can take twenty minutes now. If a company measured juniors purely by lines of working code produced, the maths would look bad for us. But that was never really what juniors were for. What a junior is actually for Teams hire juniors because juniors become seniors. The first year is an apprenticeship: you do small, low-risk work, someone reviews it, you learn how the system and the team actually work, and slowly you're trusted with more. The output in that first year was always modest. The value was in what the person turned into. If companies stop hiring juniors because AI covers the modest output, they'll find in a few years that there's nobody to promote. I expect some will make that mistake anyway, because hiring is often decided quarter by quarter. That's a real risk for people my age, and I don't want to wave it away. What doesn't get automated The parts of the job I find hardest aren't typing code. They're things like: - Working out what the problem is. Requirements are usually incomplete, and the person asking often doesn't know what they need yet. - Knowing the system. Why this table has a strange column, why that service can't be called during a batch run, which team owns what. - Judgment. Is this change safe? Is this abstraction worth it? Should this be built at all? - Debugging things that don't fit in a prompt. A failure that only shows up with production data, across three services, at month end. - Owning the outcome. Someone has to be accountable when it breaks. An assistant can help with each of these. It can't do them for you, because they depend on context it doesn't have and responsibility it can't hold. The real risk: skipping the learning The thing I actually worry about isn't replacement. It's that students and juniors use these tools to skip the part where you learn. If an assistant writes my code and I paste it in without understanding it, I've shipped something, but I haven't got any better. Do that for a year and I'm a person who can prompt but can't reason about what comes back. That person is genuinely replaceable, because the only skill they've built is the one the tool already has. The uncomfortable truth is that struggling with a bug for an hour is often where the learning happens. Tools that remove the struggle also remove the lesson, unless you're deliberate about it. How I'm trying to learn now I don't think the answer is to avoid the tools. Refusing to use them would be like refusing to use Stack Overflow ten years ago. What I'm trying to do instead: - Write it myself first when I'm learning something new. Then compare with what the assistant suggests. The diff teaches me more than either version alone. - Never commit code I can't explain. If I can't walk through it line by line, it doesn't go in. - Read more code than I generate. Open-source projects, library internals, other people's pull requests. Reading builds the judgment that lets you spot when generated code is subtly wrong. - Ask the tool "why", not just "how". Then check the answer against the documentation, because it's sometimes confidently wrong. - Learn the fundamentals properly. Data structures, databases, networking, how memory and concurrency actually behave. These are what let you evaluate output instead of trusting it. So, will it take junior jobs? Some of them, probably. I think the number of roles that consist of turning clear tickets into straightforward code will shrink. The bar for entry will move up: a junior will be expected to do what a junior plus a year of experience did before, because the tools cover the gap. I don't think the junior role disappears, because the need for people who grow into seniors doesn't disappear. What changes is what a junior needs to bring: less raw typing speed, more understanding, more ability to check and question what a tool produces, and more of the communication and ownership that used to come later. That's a harder start than the one people a few years ahead of me had. It's also, honestly, a more interesting one. Where this leaves me - Use the tools, but don't let them do your learning for you. - Don't ship what you can't explain. - Put real effort into fundamentals and reading code; that's where judgment comes from. - Look for teams that still invest in reviewing and mentoring juniors. An apprenticeship is worth more than a slightly higher first salary. I'd rather be the person who knows when the generated code is wrong than the person who can only generate it. Update, October 2026: I revisited this after Claude Opus 5.5 came out, in Opus 5.5 and the Junior Question, Seventeen Months Later.
The Chinese Room, Revisited After ChatGPT
I first came across the Chinese Room as a neat puzzle from 1980. Then I spent two years watching chatbots get good enough that my classmates now argue about whether they "understand" anything, and the puzzle stopped feeling academic. This post is me working through the argument again, with a large language model open in another tab. I don't reach a verdict. I'm not sure anyone honestly can yet. The argument John Searle asks you to imagine a person who speaks no Chinese, locked in a room. Slips of paper with Chinese characters come in through a slot. The person has a huge rulebook, written in English, that says: when you see these symbols, write down those symbols and pass them back out. The rulebook is good enough that the people outside, who do speak Chinese, are convinced they are talking to a fluent speaker. Searle's point: the person in the room doesn't understand a word of Chinese. They are just matching shapes. And a computer running a program is in the same position as that person. It manipulates symbols according to syntax (rules about form), and syntax alone is never enough for semantics (meaning). So passing a conversation test, however convincingly, doesn't show understanding. The target was what he called strong AI: the claim that a suitably programmed computer literally has a mind, rather than merely simulating one. The systems reply The most common response, and the one I find most tempting, is the systems reply. Of course the person doesn't understand Chinese. Neither does a single neuron in your head understand English. Understanding, if it exists, belongs to the whole system: the person, the rulebook, the paper, the process. Searle's answer was blunt. Let the person memorise the entire rulebook and do everything in their head. Now they are the whole system, and they still don't understand Chinese. So the systems reply fails. I've never found that fully convincing. Someone who had genuinely internalised a rulebook capable of fluent Chinese conversation would be running something very unlike "following instructions" as we normally picture it. Our intuitions about what that person would or wouldn't understand are built for small, slow cases. The thought experiment asks us to trust them at a scale where they may simply not apply. But "your intuition might be wrong" is not the same as "you are wrong". What changes with LLMs A large language model is, in a loose sense, a Chinese Room you can actually talk to. It takes in tokens, applies a vast set of learned numerical rules, and produces tokens. It has no eyes, no body, and no direct contact with the things its words refer to. In Searle's terms it looks like pure syntax. A few things feel different, though. The rulebook wasn't written by anyone. Searle's room has a hand-written rulebook, which makes it easy to picture as a lookup table. An LLM's "rules" are billions of parameters learned from text. Nobody wrote them, and we don't fully know what structure they encode. Interpretability research has found internal representations that track things like syntax, sentiment, and relations between entities. Whether that counts as anything like meaning is exactly the open question, but it isn't a lookup table. The behaviour generalises. These models handle prompts nobody has written before. They translate, explain jokes, write code for problems that don't appear in their training data verbatim. That makes the room harder to dismiss as a trick. The failures are strange. The same model that explains a subtle proof will confidently invent a citation or fumble a simple counting task. To me that looks less like a mind and more like a very powerful pattern engine with no stable sense of what is true. What doesn't change The core of Searle's argument is untouched by any of this. His claim was never "computers can't produce fluent behaviour". It was "fluent behaviour, however good, doesn't establish understanding". ChatGPT makes the behaviour far more impressive. It doesn't make the inference from behaviour to understanding any more valid. If anything, LLMs sharpen the problem. We now have a system that passes many informal conversation tests and that most of its builders would hesitate to call conscious. So we are forced to ask what, beyond behaviour, we would want as evidence. I don't have a good answer, and I notice I don't have one for other people either. I assume my friends understand things because they are built like me, not because I've checked. Where I've landed, for now Some honest positions I hold loosely: - "It's just predicting the next token" is true and not decisive. Describing the mechanism doesn't settle what the mechanism amounts to. "It's just neurons firing" is also true of us. - "It obviously understands" is also too fast. Fluency is what these systems are optimised for. Being impressed by fluency is the one test they were built to pass. - The word "understand" is doing too much work. It bundles together reliable use of concepts, grounding in the world, and subjective experience. LLMs may have some of the first, little of the second, and we have no idea how to measure the third. Splitting the word apart is the most useful thing I've taken from rereading Searle. The question "does it understand?" is almost unanswerable. Questions like "does it track the truth?", "does it connect words to the world?", and "is there anything it is like to be it?" are at least ones we can argue about separately. The short version If you build things with these models, the practical stance is simple: treat the output as fluent text that needs checking, not as the words of something that knows what it's saying. That holds whichever side of the philosophy turns out to be right. The philosophical question is still open, and I'd be suspicious of anyone, on either side, who tells you it's settled.