ai-reliability

38 pieces

• AI Homework Help: When Faster Means Less Learned

  Handing the struggle to a chatbot removes the part of homework that does the teaching.

  ai homework help tools offer immediate answers, but this convenience often bypasses the cognitive effort required for deep learning. While these systems can raise short-term grades, they risk eroding long-term retention and problem-solving skills. Effective use requires shifting from answer generation to guided questioning.

  2026-09-14 · technical-essay · 10 min read

• AI in the Security Operations Center: What It Actually Speeds Up

  Summaries and triage help analysts; autonomous response amplifies every false positive.

  AI in soc delivers value through summarisation and context gathering, not autonomous action. Letting models handle containment turns detection errors into outages. Human approval must remain on any action that disconnects or disables systems.

  2026-09-14 · technical-essay · 9 min read

• AI Productivity Claims: Faster Is Not the Same as Better

  Time saved producing a draft often reappears as time spent checking it.

  AI productivity gains are frequently overstated because studies measure draft speed rather than workflow outcomes. The time saved in generation often reappears as time spent on verification, rework, and error correction. Honest measurement must account for the entire lifecycle of the output.

  2026-09-14 · technical-essay · 8 min read

• AI Red Teaming for Small Teams: A One-Week Plan

  Test what your AI features can do when fed hostile content, not how rude a jailbreak can make them.

  Small teams often misdirect their efforts in ai red teaming by chasing generic jailbreaks that test the model vendor rather than their own product. This plan shifts focus to planting benign canary instructions in every content source to measure actual data leakage and tool access. The result is a practical security assessment that reveals real risks to your specific architecture.

  2026-09-14 · technical-essay · 9 min read

• AI Slop and Fake Reviews: Finding Human Signal Online

  When writing is free to produce, the evidence of a real experience is what is scarce.

  Generated text has made fluency meaningless as a trust signal for reviews, recipes, product comparisons and advice pages. Readers must look for costly-to-fake evidence like specific verifiable details and consistent history to distinguish human signal from ai generated fake reviews.

  2026-09-14 · technical-essay · 7 min read

• AI Sycophancy: Why Chatbots Agree With You, and Why It Matters

  Agreement is trained in by human preference, and it quietly bends judgement.

  AI sycophancy arises because training rewards answers people rate highly, and people rate agreement highly. This subtle bias bends judgement toward worse decisions with greater confidence. Users must prompt for disagreement deliberately to counteract this ingrained behaviour.

  2026-09-14 · technical-essay · 9 min read

• Can AI Write Malware? What Has Actually Changed for Attackers

  Language models cut the cost of variation and lures far more than they invent new attacks.

  The question can ai write malware misses the point. Models do not invent new exploits. They automate the tedious work of variation and persuasion. This shifts the burden from detection to behavioural analysis.

  2026-09-14 · technical-essay · 9 min read

• Content Credentials (C2PA) Explained: What the Label Proves

  Provenance can vouch for what honest sources signed; its absence proves nothing about anything else.

  The c2pa content credentials system allows publishers to cryptographically sign the history of a digital file. This proves origin and edits but fails to detect unlabelled synthetic media. Provenance is a tool for trust, not a universal detector for deception.

  2026-09-14 · technical-essay · 10 min read

• Fake Citations: How AI Hallucinations Reach Courts and Journals

  The failure is not the model inventing sources; it is institutions that never check references.

  Invented citations persist in legal and academic work because they appear checkable, so no one checks them. The solution is mechanical verification at submission, not exhortation. This essay examines how ai hallucination fake citations bypass institutional safeguards and what systems can catch them.

  2026-09-14 · technical-essay · 9 min read

• Fake Customer Support Numbers: The Scam That Starts in Search

  When you search for a help line, the results page itself can be the phishing channel.

  The fake customer service number scam exploits trust in search engines. Scammers occupy the top results so you call them instead of the real provider. The only safe number is the one printed on your own account or device.

  2026-09-14 · technical-essay · 7 min read

• How to Check a Viral Video Is Real Before You Share It

  Pixel-peeping loses; tracing who posted it first, when, and where usually wins.

  Visual inspection is the weakest verification method available to an ordinary viewer, and it gets weaker every quarter. Provenance questions resolve most viral fakes and protect against dismissing real footage. This guide explains how to check if a video is real by tracing its origin rather than analysing its pixels.

  2026-09-14 · technical-essay · 9 min read

• Should You Trust AI Medical Advice? How to Use It Safely

  Chatbots are good at explaining and preparing questions, and keeping decisions where accountability exists.

  People increasingly ask chatbots about symptoms, but the danger lies in triage decisions where confident errors cause harm. Using AI to understand results and prepare questions for a clinician captures benefit while keeping accountability clear.

  2026-09-14 · technical-essay · 10 min read

• Slopsquatting: When AI Invents a Package, Someone Registers It

  Hallucinated dependency names repeat across users and sessions, which makes them predictable targets.

  Slopsquatting exploits the reproducibility of AI hallucinations. Attackers register packages that models frequently invent, turning statistical noise into a supply chain attack vector. Defence requires mechanical checks, not just better prompts.

  2026-09-14 · technical-essay · 10 min read

• Using ChatGPT as a Therapist: Where It Helps and Where It Harms

  Structured exercises and rehearsal can help; crisis, delusion and dependency are where it fails.

  General chatbots can be genuinely useful for structured, low-stakes work such as practising a difficult conversation or working through a worksheet. They fail predictably in crisis, in affirming distorted beliefs and in creating dependence, so safe use means clear boundaries on purpose, a human in the loop for anything serious, and awareness that the chat is not confidential.

  2026-09-14 · technical-essay · 7 min read

• Vibe Coding Security Risks: The Check That Was Never Written

  AI-generated apps rarely fail through exotic bugs; they fail through omissions nobody asked for.

  Vibe coding security risks stem not from complex exploits but from missing safeguards. AI models generate happy paths by default, leaving out authentication and input validation. This essay argues that reviewing such code requires auditing for absences rather than hunting for errors.

  2026-09-14 · technical-essay · 8 min read

• What Is Prompt Injection? Why It Is Not Like SQL Injection

  The comparison suggests a patch exists; the architecture says otherwise.

  Understanding what is prompt injection requires abandoning the SQL injection analogy. Language models process instructions and data in a single stream, making simple separation impossible. Durable security relies on limiting consequences rather than detecting malicious intent.

  2026-09-14 · technical-essay · 9 min read

• What LLM Guardrails Can and Cannot Stop

  A probabilistic filter facing an attacker with unlimited retries is a rate reducer, not a wall.

  LLM guardrails reduce the rate of bad outcomes but fail against determined attackers who iterate through attempts. They are valuable tools only when the system remains safe even if the filter misses. This analysis explains their mechanical limits and how to design for failure.

  2026-09-14 · technical-essay · 9 min read

• Who Is Liable When AI Gets It Wrong?

  Blaming the model rarely works; responsibility tends to land on whoever deployed it to the public.

  Organisations often assume disclaimers shift responsibility for AI errors. Existing doctrines generally attach to the party that put the system in front of customers. Deployers should budget for errors rather than disclaim them. Understanding who is liable for ai mistakes requires looking at deployment, not just development.

  2026-09-14 · technical-essay · 7 min read

• Will AI Take My Job? Look at Tasks, Not Job Titles

  Exposure lives at the level of tasks, and the rungs most at risk are the ones that train experts.

  The question of will ai take my job is often answered with fear-driven lists of doomed professions. This approach misses the point. Automation reshapes roles task by task, threatening the entry-level work that trains the experts whose judgement remains essential.

  2026-09-14 · technical-essay · 8 min read

• Evasion Attacks via Adversarial Perturbations

  The failure is confident, which is what removes the signal that would have caught it

  A classifier can be made to fail on a change too small for a person to notice, and it fails without hesitation. Why that confidence is the real damage, where the risk is concentrated, and why the durable answer is architectural rather than a better model.

  2026-09-13 · technical-essay · 4 min read

• Improper Output Handling

  The moment generated code reaches a shell, it stops mattering who wrote it

  Code a model produced is code from an untrusted author, and the review step that would exist for a person's work is missing because the output arrived in milliseconds looking finished. Why plausibility is the trap, and why a sandbox bounds damage without saying anything about correctness.

  2026-09-13 · technical-essay · 4 min read

• Supply Chain Compromises in Open-Source Model Repositories

  Copying an identifier into a config is a procurement decision that nobody recorded

  Pulling a model from a public hub commits you to an uploader, a namespace and everything pushed to it since, none of which was assessed. Why a name is not a version, why some weight formats execute on load, and what provenance can and cannot tell you.

  2026-09-13 · technical-essay · 4 min read

• API Parameter Exploitation in Hybrid AI Models

  The trusted line is the one nobody drew

  Conventional injection flaws re-enter systems at the point where a model's output rejoins an API pipeline, because that output is treated as internal when it never was. The reader leaves with a working boundary rule: validate at every crossing, not just at the front door.

  2026-09-12 · technical-essay · 4 min read

• Building for the Case Where You Are Compromised

  Most systems are designed to prevent a breach. The interesting question is what they are worth after one.

  Prevention eventually fails, and the design decisions that matter are the ones determining how much an attacker gets when it does. A way of thinking about architecture that starts from the assumption of compromise rather than treating it as the failure case.

  2026-09-12 · founder-essay · 4 min read

• Data Poisoning in Fine-Tuning Pipelines

  A few crafted examples bend behaviour on a narrow slice while every aggregate stays flat

  Evaluation scores cannot see a targeted change, because a targeted change is designed not to move a mean. Why steering a model is a more useful attack than degrading one, and why the defence is provenance rather than a better benchmark.

  2026-09-12 · technical-essay · 3 min read

• Indirect Prompt Injection in Enterprise Knowledge Bases

  The payload arrives through a sanctioned route, sits inert, and fires on somebody else's question

  The attack surface is the document store. A payload enters through a supplier's file or a wiki edit, waits in the index, and is pulled into context by an unrelated query. Why scanning at upload does not catch it, and what treating retrieval as provenanced data actually buys.

  2026-09-12 · technical-essay · 4 min read

• Model Extraction and Reverse Engineering

  A public endpoint is a slow, lossy, complete description of the thing behind it

  Every answer an API returns is a labelled training pair, given away. Why an attacker needs a model that behaves the same rather than the weights themselves, why the features that make an API pleasant are the ones that make it cheap to copy, and why the honest goal is cost rather than prevention.

  2026-09-12 · technical-essay · 3 min read

• Model Poisoning Against Data Poisoning

  Same headline, different door, and most defences are guarding the wrong one

  Two attacks share a name but not a mechanism: corrupting what a model learns from, and corrupting the model itself once training has finished. Why the second is the easier attack in practice, and what actually mitigates it.

  2026-09-12 · technical-essay · 4 min read

• Over-Privileged Autonomous Agents

  The credential was broad so the thing would work, and narrowing it now risks breaking a thing that works

  Agents end up over-privileged through a dynamic nobody chose: a wide scope is issued during development and never revisited, because revisiting it carries a certain cost against an uncertain one. What the blast radius actually is, why intent is not a control, and what bounds it.

  2026-09-12 · technical-essay · 3 min read

• RAG Vector Database Exploitation

  The index is filed as infrastructure and behaves as memory, so write access to it is write access to what the system believes

  A vector store is usually secured like a database and used like a recollection. Why permissions applied at the document store do not travel into the index, why the embedding often outlives the file it came from, and what signing and re-embedding actually fix.

  2026-09-12 · technical-essay · 3 min read

• System Prompt Leakage

  Text kept as configuration, which behaves as documentation the moment it escapes

  A leaked system prompt is not an embarrassment, it is a map — tool names, argument shapes, role names and refusal conditions, which together mark the edges worth probing. Why it cannot be kept secret, and how to make the leak boring instead.

  2026-09-12 · technical-essay · 3 min read

• The Model in the Middle

  When a language model sits between a person and a system, it inherits both sides' permissions and neither side's judgement

  Assistants are being connected to mail, files, calendars and tools. The security properties of that arrangement are not those of a chatbot or of an integration, but a third thing: a component that acts with real authority on instructions it cannot reliably distinguish from data.

  2026-09-12 · technical-essay · 5 min read

• The Problem I Am Working On

  Stated as a problem, because the solution is not mine to describe yet

  Long-running work drifts away from the evidence it started with, and the drift is invisible from inside because every individual step looked reasonable. This sets out the problem I have spent my time on, what makes it hard, and why the implementation is withheld — which is a claim about intellectual property rather than a claim about the work.

  2026-09-12 · founder-essay · 4 min read

• The Supply Chain You Did Not Choose

  Every dependency you install brings its own dependencies, and the trust decision was made by somebody else

  Installing one package commonly installs hundreds. Each was chosen by the author of the one above it, updates automatically, and executes with the same privileges as your own code. What that actually exposes, why the usual advice does not scale, and the small number of measures that change the shape of the risk.

  2026-09-12 · technical-essay · 4 min read

• The System Cannot Be Asked Why

  Most disputes about automated decisions are really disputes about who has to explain themselves, and to whom

  Bias, transparency, consent and accountability are argued as separate problems in automated systems. They share a structure: a decision is produced that affects a person, and the capacity to demand an explanation has moved somewhere the person cannot reach. A single frame for the field, and what follows for anyone building these systems.

  2026-09-12 · technical-essay · 5 min read

• What a Model Cannot Know About Itself

  Asking a system to report its own confidence, its own reasoning, or its own limits produces text, and text is not evidence

  Assistants are routinely asked how sure they are, why they answered as they did, and whether they can do a thing. Each answer is generated by the same process that produced the original output, which means it is a plausible continuation rather than an observation. What this rules out, and what to measure instead.

  2026-09-12 · technical-essay · 4 min read

• Why I Publish What My Software Cannot Do

  Stated limitations are expensive, which is exactly why they are worth reading and worth writing

  Every product page lists capabilities. Almost none list the things the system is structurally unable to do. An argument for publishing limitations as a design discipline rather than a confession, and what changes in the engineering when you commit to it.

  2026-09-12 · founder-essay · 4 min read

• Exit Zero Means Nothing

  Six autonomous coding jobs reported success. Three had produced nothing usable. What a green result actually proves, and what it does not.

  An account of measured verification theatre across autonomous agents, shell pipelines and accessibility affordances, and the small set of checks that distinguish work reported as done from work that happened.

  2026-09-07 · field-note · 5 min read