When an AI system gives a bad answer, teams often look first at the model or the prompt. For government services, that can miss the more important failure: the data pathway may no longer be trustworthy.
An AI assistant connected to policies, case material, website content or operational records depends on more than a capable model. It depends on where its data came from, who changed it, how it was transferred, what was allowed into the retrieval index, and whether the live information still resembles the data used during evaluation.
The Australian Signals Directorate's Australian Cyber Security Centre now brings these concerns together in its AI data security guidance. The joint publication focuses on three risks across the AI lifecycle: the data supply chain, maliciously modified or poisoned data, and data drift. That makes data engineering a security and service-reliability concern, not merely a preparation step before the model is switched on.
The Model Is Only One Dependency
Modern AI services rarely operate on a sealed training dataset. A government use case may combine a hosted model with document repositories, search indexes, content-management systems, APIs, user feedback and periodically refreshed reference material.
Each connection expands the data supply chain. A retrieved document might be authoritative, obsolete, duplicated, incorrectly classified or deliberately manipulated. A connector may acquire broader access than the service needs. A scheduled import may silently fail while the AI keeps answering from stale content. User feedback may be useful evidence, or it may become an unreviewed route for changing future behaviour.
The ACSC guidance makes the underlying principle clear: machine-learning systems learn or derive behaviour from data, so an attacker who can manipulate the data can influence the system's logic or outputs. Protecting the model endpoint while leaving the data pipeline opaque is therefore incomplete security.
Three Data Risks to Design For
1. Data supply-chain risk
AI data may pass through publishers, vendors, collection tools, transformation jobs, object stores, vector databases and application services before it reaches the model. Teams need to know which sources are authoritative, which processing steps occurred, who or what approved each revision, and whether integrity was preserved in storage and transit.
2. Poisoned or maliciously modified data
Poisoning is not limited to training a foundation model. In a retrieval-augmented system, a manipulated page or document can be indexed and repeatedly presented to the model as trusted context. The malicious content may be factually false, may redirect the system's behaviour, or may attempt to influence connected tools.
3. Data drift
Even honest data changes. Policies are replaced, terminology shifts, services move, users behave differently and source systems change structure. A system that performed well against last quarter's material may become less accurate or less useful without any code or model release. Drift is an operational condition that must be detected and assessed.
Build a Data Bill of Materials
Software teams increasingly keep a software bill of materials to understand components and dependencies. An AI service benefits from a comparable, practical inventory for its data.
For each source or corpus, record:
- the authoritative owner and business purpose;
- the collection or access method;
- classification, privacy and records requirements;
- licence or permission to use the material;
- the approved version, checksum or immutable snapshot;
- transformation, redaction, chunking and enrichment steps;
- where the processed data and indexes are stored;
- which service identities can read, change or approve it;
- refresh frequency, expiry conditions and rollback method;
- the evaluation set that shows the source remains fit for its intended use.
This does not need to become a ceremonial spreadsheet. It can be implemented as version-controlled metadata beside ingestion code, signed manifests for released corpora, pipeline logs and deployment evidence. The goal is to answer a basic incident question quickly: which exact data influenced this version of the service?
Do Not Promote Raw Content Straight Into Retrieval
A safe ingestion pipeline should separate collection from use. New material lands in a quarantine area, passes deterministic checks, receives the required human or system approval, and only then becomes available to the production retrieval service.
Useful controls include:
- allowlisting authoritative source locations rather than crawling broadly;
- malware scanning and file-type validation before parsing;
- schema, size, language and duplication checks;
- detection of unexpected embedded instructions or hidden content;
- privacy and classification checks before indexing;
- cryptographic hashes for source and processed artefacts;
- review of material changes to high-authority sources;
- a clean promotion step from candidate index to production index;
- the ability to remove one source and rebuild without losing provenance.
Prompt-injection defences are still required at runtime, but the ingestion boundary reduces the chance that unreviewed content is permanently treated as trusted organisational knowledge.
Privacy Applies to Inputs, Retrieval and Outputs
The Office of the Australian Information Commissioner says the Privacy Act applies to uses of AI involving personal information. Its guidance for commercially available AI products notes that obligations can apply both to personal information entered into a system and to generated output containing personal information. Inferred, incorrect or artificially generated information about an identifiable person may also be personal information.
That has practical consequences for retrieval architecture. A team should not assume that splitting documents into embeddings, moving them into a vector store or sending selected chunks to a model removes their privacy character. Access controls should follow the underlying information, retrieval should enforce the caller's permissions, and logs should avoid collecting unnecessary prompts or retrieved content.
The OAIC also recommends privacy-by-design assessment and warns against a set-and-forget approach to AI products. Product due diligence, human oversight, staff training and ongoing monitoring all remain relevant after procurement.
Measure the Data Path, Not Just Answer Quality
Evaluation needs to identify whether a failure came from the model, retrieval, the corpus or the service around them. A single aggregate accuracy score cannot do that.
A useful operating dashboard can measure:
- freshness: expected sources updated on time and obsolete versions removed;
- coverage: important user questions have current authoritative material available;
- retrieval quality: relevant evidence appears in the top results without crossing access boundaries;
- grounding: answers remain supported by retrieved sources and cite the right revision;
- integrity: source and processed hashes match approved manifests;
- drift: changes in queries, content, retrieval scores, abstention rates and correction patterns;
- incidents: suspected poisoning, unauthorised access, stale-source use and unsafe output are triaged and recoverable.
When a source changes materially, re-run representative evaluations before promotion. The ACSC guidance says verification and validation are necessary when new data or user feedback is introduced. Treating a corpus update like a controlled release gives teams a repeatable way to satisfy that expectation.
Turn Government Trust Requirements Into Release Evidence
Criterion 5 of the Australian Government Digital Service Standard asks agencies to adopt transparent data handling, implement security measures, maintain a reliable service and remain accountable for it. Those responsibilities fit naturally into the AI data pipeline.
A production release should be able to show:
- the approved data inventory and intended purpose;
- privacy, security, classification and records decisions;
- source provenance and integrity evidence;
- ingestion and retrieval tests against representative scenarios;
- the accountable owner for corpus changes and incidents;
- monitoring thresholds, rollback steps and the non-AI fallback;
- what users are told about data collection, use, storage and AI involvement.
This complements the stage gates in our guide to moving AI from proof of concept to production. A pilot should test the real data pathway under controlled conditions; production should prove that the pathway can be operated, monitored and repaired.
A Practical Four-Week Data-Security Sprint
- Week 1 — map the data supply chain. Inventory every source, connector, transformation, store and recipient. Identify the owner, authority, classification and current gaps in provenance.
- Week 2 — create a controlled ingestion path. Add quarantine, validation, versioning, least-privilege access, integrity checks and an explicit promotion step.
- Week 3 — build representative evaluation. Test retrieval, grounding, permissions, stale documents, malicious content and service behaviour when a source or model is unavailable.
- Week 4 — rehearse change and recovery. Promote a corpus update, detect a failed or poisoned source, remove it, rebuild the index, restore the last approved version and capture the evidence.
The result should be a decision, not just a diagram: which use cases are ready, which sources need repair, what controls are mandatory, and where AI should remain outside the service.
What This Means for GovCMS and Drupal Teams
GovCMS and Drupal are often authoritative publishing systems, but AI integrations may copy content into separate indexes and services with different permissions and update cycles. The public website can be correct while the assistant continues answering from an older or incomplete corpus.
Teams should preserve content revision IDs, publication status, canonical URLs and access rules through ingestion. Unpublished drafts, form submissions and restricted editorial material should not enter a public-facing retrieval index. A content unpublish or correction event should also remove or supersede the indexed copy, with a test proving the change reached the AI service.
The same discipline applies to AI-assisted editorial workflows. Before generated content reaches GovCMS or Drupal, preserve source attribution, require human review, run accessibility and link checks, and keep the final publishing authority separate from generation.
Working with NAITEC Digital
NAITEC Digital helps Australian Government and business teams build AI services whose data pathways are as testable as their code. We can map the data supply chain, design controlled ingestion and retrieval, connect privacy and security requirements to delivery evidence, and implement monitoring, evaluation and recovery.
We are a Newcastle, NSW software consultancy, a BuyICT registered supplier, and GovCMS/Drupal specialists on the Drupal Services Panel. Our capabilities span AI integration, automation and custom software delivery and government digital services, GovCMS and Drupal.
If your AI pilot can answer questions but cannot yet prove which data it trusted, talk to NAITEC Digital. We can help turn the data pipeline into a controlled, measurable part of the service.
Frequently Asked Questions
What is AI data poisoning?
It is the deliberate modification or introduction of data intended to influence an AI system's behaviour or outputs. The risk can affect training and fine-tuning data, user feedback, or documents promoted into a retrieval system.
Does retrieval-augmented generation remove the need for data governance?
No. Retrieval can make sources more current and inspectable, but it creates a live data supply chain that needs provenance, access control, validation, monitoring and controlled updates.
How should teams detect data drift?
Monitor source changes, query patterns, retrieval quality, grounding, abstention and correction rates against a representative baseline. Re-evaluate the system when important data or user behaviour changes.
Can personal information be stored in a vector database?
Architecture decisions depend on purpose, authority, risk and applicable law. Moving information into embeddings or a vector store does not by itself remove privacy obligations. Apply data minimisation, access controls, security safeguards, retention rules and privacy assessment.
Can NAITEC Digital help secure an existing AI data pipeline?
Yes. NAITEC Digital can assess sources and connectors, design controlled ingestion, test retrieval and permissions, establish versioned release evidence, and implement monitoring and rollback. Contact us to discuss the service.