Senior Test Analyst

We are seeking an experienced Senior Test Analyst to support a portfolio of AI-centric projects focused on agentic AI, AI governance, and intelligent automation solutions. The role will involve designing and executing scenario-based testing, validating AI-driven workflows, assessing AI outputs, and contributing to quality assurance across integrated systems and emerging AI technologies.

The successful candidate will work closely with business, technical, and delivery teams to ensure AI solutions are robust, reliable, ethical, and aligned with organisational objectives and responsible AI principles.

Employment Type and Duration

• Contract - Initial 12 months with potential extensions

Location

• Brisbane City, QLD - Hybrid

Key Responsibilities

• Design and execute scenario-based testing for AI-enabled solutions across multiple projects
• Develop varied test inputs to assess AI system behaviour under different conditions
• Evaluate AI outputs for relevance, consistency, clarity, appropriateness, and reliability
• Perform prompt and input variation testing to assess robustness and sensitivity
• Conduct exploratory testing to identify edge cases, emergent behaviour, and integration issues
• Validate AI components within end-to-end workflows and integrated enterprise systems
• Log, categorise, and triage defects including behavioural observations and integration issues
• Capture structured test evidence, findings, limitations, and risk observations
• Contribute to test planning, acceptance criteria definition, and test data preparation
• Support automation of integration and end-to-end testing within CI/CD pipelines Required Skills & Experience
• Proven experience as a Test Analyst in complex enterprise system environments
• Hands-on experience testing AI, machine learning, or data-driven systems
• Experience testing agentic AI systems and autonomous AI behaviours
• Familiarity with AI frameworks such as LangChain, CrewAI, and AutoGen
• Strong experience in API, integration, and end-to-end testing
• Exposure to prompt engineering and scenario-based testing methodologies
• Knowledge of AI-specific risks including bias, hallucinations, explainability, and model variability
• Experience with observability and AI monitoring tools such as Langfuse, Maxim AI, and Arize AI
• Strong analytical, problem-solving, and observational skills
• Experience using JIRA, Zephyr Scale, Confluence, and CI/CD pipelines Qualifications & Clearance
• ISTQB Certified Tester - AI (CT-AI) or equivalent certification highly desirable
• Experience with LLM red teaming and familiarity with OWASP AI security frameworks desirable
• Relevant tertiary qualification in Information Technology, Computer Science, or related discipline preferred
• Ability to provide recent and verifiable experience on AI-centric projects If you have what it takes to thrive in this role and want to know more, please reach out to Isha Khanna at ***email_hidden*** Privacy Due to new privacy laws regarding the collection and handling of personal information, all applications must be submitted online. Once you click on the "apply for this job" button below you will be directed to our Online Centre where you will have the opportunity to read and agree to our Privacy and Consent statements and it will allow your details to be formally lodged and processed correctly. "Akkodis is an Equal Opportunity employer, committed to all forms of workforce inclusion. We actively invite and encourage applications by candidates from diverse backgrounds"