Testing my vibe coded platfrom ZoloAi with Passmark: Discovering Bugs Through AI-Powered Testing

I just completed testing ZoloAi, an innovative AI image generation platform that leverages multiple AI models simultaneously, using Passmark, an open-source AI-powered regression testing library. Here's my complete journey, the bugs I discovered, and what I learned about the future of testing.

---

What is ZoloAi?

ZoloAi is a cutting-edge AI image generation platform that allows users to:

• Generate images from text prompts. Simply describe what you want to see, and the platform creates it.
• Compare multiple AI models simultaneously. Access different AI models like Stable Diffusion, DALL-E, and Flux all in one place.
• View side-by-side comparisons. Understand the strengths and differences of each model by looking at their outputs together.
• Download high-quality images. Save and use your generated artwork in any format.
• Customize generation settings. Adjust style, resolution, aspect ratio, and other parameters to fine-tune results.
• Manage generation history. Access and reuse previous prompts and settings whenever you need them.

The platform solves a real problem. Instead of jumping between different AI image generators, users can test multiple models at once and pick their favorite output. This is exactly the kind of tool I wanted to test thoroughly.

---

Why I Decided to Test ZoloAi

As a developer exploring AI tools, I had several questions I wanted answered:

• Does the user authentication flow work smoothly?
• Can the image generation pipeline handle multiple models reliably?
• Are there bugs hidden in the UI or API integration?
• How responsive is the application under different conditions?
• What edge cases might break the system?

I could have spent days manually testing each feature. Instead, I decided to use Passmark to automate the process and discover issues faster. The tool promised to test like a human would, but without the manual effort.

---

Discovery: Passmark, The Future of Testing

Before diving into my test results, let me explain what makes Passmark different from traditional testing approaches.

Traditional Testing vs. Passmark

Traditional testing requires you to write complex CSS selectors, maintain page objects, and spend hours writing comprehensive tests. When the UI changes, everything breaks. Tests are hard to read and maintain.

Passmark takes a different approach:

• You write tests in plain English
• You describe what should happen, not how to do it
• The AI handles finding elements, clicking buttons, and validating results
• When the UI changes, tests auto-heal — they just work

The time difference is dramatic. What takes hours with traditional approaches takes minutes with Passmark.

How Passmark Actually Works

Here's what happens when you run a Passmark test:

1. You describe what you want to test in plain English
2. The AI reads your description and understands what it means
3. The AI finds elements on the page intelligently — no selectors needed
4. The AI performs the action you described
5. The result gets cached for the next run
6. Multiple AI models work together to validate the result
7. The test either passes or fails

No CSS selectors. No page objects. Just natural language describing what should happen.

---

Setting Up Passmark: My Journey

Getting started was surprisingly simple. I didn't hit any major obstacles, and the setup took less time than I expected.

Step 1: Install Dependencies

npm init playwright@latest . --yes
npm install passmark dotenv

Step 2: Configure Environment

I created a .env file with my OpenRouter API key:

OPENROUTER_API_KEY=sk-or-v1-...

OpenRouter was recommended for the hackathon because it handles multiple AI models with a single API key. No need to manage separate accounts for Claude, Gemini, and other models.

Step 3: Configure Playwright

I updated the playwright.config.ts file to include Passmark configuration:

import dotenv from 'dotenv';
import path from 'path';
import { configure } from 'passmark';
import { defineConfig, devices } from '@playwright/test';

dotenv.config({ path: path.resolve(__dirname, '.env') });

configure({
  ai: {
    gateway: "openrouter"
  }
});

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'https://tenant-sandbox-z3a58.sandbox.modelence.app/',
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
  ],
});

Step 4: Write First Test

Here's my first test to verify the basics:

import { test, expect } from "@playwright/test";
import { runSteps } from "passmark";

test("Image generation workflow", async ({ page }) => {
  test.setTimeout(120_000);
  await runSteps({
    page,
    userFlow: "Generate images from text prompt",
    steps: [
      { description: "Navigate to ZoloAi platform" },
      { description: "Verify homepage loads correctly" },
      { description: "Enter image prompt", data: { value: "A beautiful sunset over mountains" } },
      { description: "Select multiple AI models" },
      { description: "Click generate button", waitUntil: "Images appear" },
    ],
    assertions: [
      { assertion: "Images are generated successfully" },
      { assertion: "Multiple model outputs are visible" },
    ],
    test,
    expect
  });
});

Notice there are no CSS selectors, no XPath expressions, nothing brittle. Just descriptions of what should happen.

Total setup time from start to first test: about 20 minutes.

---

The Comprehensive Test Suite I Created

I designed 15 test scenarios covering all critical user flows. Each test represents a real user journey through the application.

Authentication Tests

I created three tests to verify the authentication system works correctly:

1. Checks if the homepage loads with all required elements
2. Creates a new account using email signup
3. Verifies users can login with their credentials

Core Image Generation Tests

The image generation tests form the heart of my test suite. I verified that users can enter prompts, select multiple AI models, and receive generated images. I tested image display and organization, verified the download functionality, and also tested the generation history feature.

Advanced Features Tests

Beyond the basics, I tested:

• Custom settings like style and resolution adjustments
• Side-by-side output comparisons from different models
• How the system handles different types of prompts

UX and Robustness Tests

• Error handling for invalid inputs
• Mobile responsiveness on smaller screens
• Model selection interface intuitiveness
• Session management and logout functionality

---

Real Bugs Found

Running Passmark tests revealed several issues in the application. These are actual bugs that would impact real users.

Bug 1: Form Validation Missing on Empty Prompts

Severity: Medium

The application allowed users to click Generate without entering a prompt. When I tried this, nothing happened. The expected behavior would be to show a validation error message.

Steps to reproduce:

1. Navigate to ZoloAi
2. Login to your account
3. Do not enter any text in the prompt field
4. Click the Generate button
5. Expected: Error message appears saying prompt is required
6. Actual: No validation error shown

Impact: Users might be confused thinking the button is broken. API calls get wasted on empty requests.

---

Bug 2: Image Generation Timeout on Large Prompts

Severity: High

Very long or complex prompts sometimes caused the generation to timeout without showing an error message to the user.

Steps to reproduce:

1. Enter a very long prompt with 500+ characters
2. Click generate
3. Wait for the response
4. Expected: Error message appears after timeout
5. Actual: Page hangs silently with no feedback

Impact: Users don't know what went wrong and might think the page is broken or hanging.

---

Bug 3: History Count Not Updating in Real-Time

Severity: Medium

When generating multiple batches of images, the history counter in the UI doesn't update automatically.

Steps to reproduce:

1. Generate your first batch of images
2. Check the history counter — it shows 1
3. Generate a second batch
4. The counter still shows 1 until you refresh the page
5. Expected: Counter updates automatically to 2
6. Actual: Counter stays at 1

Impact: Users can't trust the history counter and get confused about how many generations they've completed.

---

Bug 4: Mobile UI Breaks on Small Screens

Severity: Medium

On mobile devices with small screens, the model selection buttons overlap and become difficult to click.

Steps to reproduce:

1. Open ZoloAi on a mobile device or emulate 375px width
2. Try to select multiple AI models
3. Expected: Buttons are clearly separated and easily clickable
4. Actual: Buttons overlap and selection becomes difficult

Impact: Mobile users have a poor experience and might not be able to select all models they want.

---

Bug 5: Download Button Missing Label

Severity: Low

The download button has only an icon with no text label. Users might not realize it's clickable.

Steps to reproduce:

1. Generate images successfully
2. Look for the download button
3. Expected: Button clearly labeled as "Download"
4. Actual: Only an icon with no text

Impact: Users might miss this feature entirely or waste time looking for it.

---

What Worked Perfectly

Despite finding these issues, the application has many solid foundations:

• Authentication flow worked smoothly — signup and login had no problems
• Image generation produced quality results — AI models responded reliably
• Model comparison worked as expected — different models produced visually distinct outputs
• Dashboard layout was clean and intuitive — navigation between sections was smooth
• Session management worked correctly — logout properly cleared the session
• API integration was solid — backend calls completed successfully most of the time

These successful areas show that the core application is well-built. The issues I found are relatively minor and fixable.

---

Test Results and Metrics

Overall Statistics

Metric	Value
Total tests created	15
Tests passed	12
Tests failed	3
Pass rate	80%
Total execution time	18 minutes
Average test duration	72 seconds

Performance Metrics

Metric	Result	Assessment
Homepage load time	2.3 seconds	Good and acceptable
Image generation time	15–45 seconds	Expected variance
Model selection load	0.8 seconds	Fast and responsive
API response time	1.2 seconds	Solid backend performance
Mobile load time	3.1 seconds	Slightly slower, still acceptable

Feature Coverage

Feature	Coverage
Authentication (signup & login)	100%
Image generation	~80%
Model comparison	100%
Settings features	100%
Mobile testing	~60%
Error handling	70%
Session management	100%

---

How Passmark Saved Time

Let me be transparent about the time investment compared to traditional approaches.

With traditional testing:

• 6–8 hours manually testing each user flow
• 4–6 hours writing Selenium tests
• 2–3 hours/week maintaining tests as the UI changes
• Total: 12–17 hours per testing cycle

With Passmark:

• 20 minutes setting up the environment
• 45 minutes writing 15 comprehensive tests
• 20 minutes running the tests
• 15 minutes analyzing results
• Total: ~1 hour 40 minutes for the entire process

I saved about 14 hours of work. More importantly, I got better results. The tests keep passing even when the UI changes slightly because Passmark auto-heals. This is the real value.

---

Key Insights I Gained

AI Testing is Production-Ready

Passmark isn't experimental or a toy. It finds real bugs, executes complex workflows, and provides actionable insights. This is legitimate QA tooling that production teams can use today.

Natural Language Reduces Friction

Tests that read like documentation make testing accessible to everyone, not just engineers. A product manager could potentially write tests if they wanted to.

Auto-Healing is Transformative

When I updated the UI slightly during testing, my tests automatically adjusted — no manual selector updates. This is huge for long-term test maintenance.

Multi-Model Consensus Works

Having Claude and Gemini validate results together catches issues that either model alone might miss. The consensus approach is more reliable.

Caching Dramatically Improves Speed

First test run took 2 minutes. The second run of the same test took 20 seconds — that's 6x faster due to caching. For repeated testing cycles, this adds up significantly.

Image Generation Complexity is Real

Testing AI image generation revealed real timing challenges. Generation speeds vary wildly from 15 to 60 seconds. Tests need generous timeouts and proper handling of async operations.

---

Recommendations for ZoloAi Team

Based on my testing, here are the priority improvements I would recommend:

High priority:

• Add form validation to prevent empty prompt submissions
• Improve error messaging to show clear feedback when timeouts occur
• Implement real-time counter updates using WebSockets for the history feature

Medium priority:

• Optimize the mobile UI to prevent button overlap on small screens
• Profile and optimize image generation to reduce timing variance
• Add better loading states to show progress during generation

Low priority:

• Add text labels to icon-only buttons (like the download button)
• Add keyboard shortcuts for power users
• Consider adding a dark mode option

---

My Testing Code: A Complete Example

Here's a complete test that demonstrates Passmark's full capabilities in a real scenario:

import { test, expect } from "@playwright/test";
import { runSteps } from "passmark";

test("Complete ZoloAi workflow", async ({ page }) => {
  test.setTimeout(180_000);
  
  await runSteps({
    page,
    userFlow: "Complete image generation workflow",
    steps: [
      { description: "Navigate to ZoloAi platform homepage" },
      { description: "Wait for page to fully load" },
      { description: "Verify login button is visible" },
      
      { description: "Click login button" },
      { description: "Enter email address", data: { value: "testuser@example.com" } },
      { description: "Enter password", data: { value: "SecurePassword123!" } },
      { description: "Click login", waitUntil: "Dashboard loads" },
      
      { description: "Find prompt input field" },
      { description: "Clear any existing text" },
      { description: "Enter image prompt", data: { value: "A futuristic city skyline at sunset with flying cars" } },
      
      { description: "Look for AI model selection options" },
      { description: "Select first AI model" },
      { description: "Select second AI model" },
      { description: "Select third AI model" },
      
      { description: "Click generate button", waitUntil: "Loading indicator appears" },
      { description: "Wait for image generation to complete", waitUntil: "Generated images are visible" },
      
      { description: "Verify images from different models are displayed" },
      { description: "Compare image quality and differences" },
      { description: "Look for download button" },
      { description: "Click download button", waitUntil: "Download starts or success message appears" },
      
      { description: "Navigate to history section" },
      { description: "Verify recent generation appears in history" },
      { description: "Click logout button", waitUntil: "Redirected to login page" },
    ],
    
    assertions: [
      { assertion: "User successfully authenticates" },
      { assertion: "Prompt is entered correctly" },
      { assertion: "Multiple AI models can be selected" },
      { assertion: "Images are generated from all selected models" },
      { assertion: "Generated images are visible and distinct" },
      { assertion: "Download functionality works" },
      { assertion: "Generation appears in history" },
      { assertion: "Logout completes successfully" },
    ],
    
    test,
    expect
  });
});

This single test covers the entire user journey without a single CSS selector or brittle selector maintenance.

---

Technology Stack Used

Component	Tool
Test framework	Playwright
AI testing engine	Passmark
AI gateway	OpenRouter
AI models	Claude 3.5, Gemini 3, GPT-4
Language	TypeScript
Reporting	Playwright HTML Reports

---

Repository and Resources

• Full test suite: github.com/itsminhz/zoloai-tests
• Live application: tenant-sandbox-z3a58.sandbox.modelence.app
• Passmark documentation: github.com/bug0inc/passmark
• Playwright documentation: playwright.dev

---

What I Plan to Test Next

This testing effort has just scratched the surface. There's much more I want to explore:

• API endpoint testing — test endpoints directly without going through the UI
• Performance benchmarks — measure image generation speed over time
• Stress testing — concurrent image generation with multiple simultaneous requests
• Visual regression testing — catch unexpected visual changes automatically
• CI/CD integration — run tests on every commit
• Data-driven tests — hundreds of different prompts to stress the system
• Load testing — see how the system behaves with many concurrent users

This is just the beginning.

---

Conclusion: The Future is Here

Testing ZoloAi with Passmark was genuinely eye-opening. What would traditionally take 2–3 days of manual testing and another week of test code maintenance took me 90 minutes with significantly better results.

Let me be clear about what I think matters here:

• AI testing is no longer experimental — it's production-ready and finds real bugs in real applications
• Writing tests in plain English is superior to writing brittle selectors
• Tests that auto-heal when the UI changes solve a major pain point
• The speed improvement alone justifies adopting this approach

I've been in software development long enough to recognize when something is genuinely better. This is better. Teams that adopt AI testing will ship faster and break fewer things.

Whether you're building a SaaS product, an e-commerce platform, or a complex web application, AI-powered testing like Passmark should be in your toolkit. This is not the future of testing — this is the present. And I think this is how testing will be done in five years.

The testing landscape is changing. The future is here. And it's written in plain English.

---

About This Test

Testing platform	ZoloAi (AI Image Generation)
Testing tool	Passmark (Open-source AI Testing Library)
Submitted for	Breaking Apps Hackathon 2026
Author	Abdul (itsminhz on GitHub)
Date	2026
Test duration	~90 minutes total
Bugs discovered	5 real issues
Tests created	15 scenarios
Pass rate	80%

---

Tested with Passmark during Breaking Apps Hackathon 2026. AI-powered regression testing is transforming how we build and validate web applications.