Unit tests are easy to write and fast to run. Production mimics perfectly controlled environments not at all. The gap between them—integration testing—is where most teams struggle.
Why Integration Tests Fail
Before fixing your tests, understand why they're broken:
Environment Differences
Your test database isn't your production database. Your test message queue doesn't have the same race conditions. Your test network doesn't have the same latency. These differences hide bugs.
State Management
Integration tests create state. Subsequent tests depend on that state—or fail because of it. Tests that pass individually fail when run together. Tests that pass locally fail in CI.
Timing Dependencies
Integrations involve network calls. Network calls take variable time. Tests that don't account for this variability become flaky.
Building Reliable Integration Tests
1. Isolate Test Data
Each test should create its own data and clean up after itself:
- Use transactions that roll back after each test
- Generate unique identifiers for test data
- Never depend on pre-existing data
- Never leave test data behind
2. Use Testcontainers
Spin up real dependencies in containers. Your tests run against actual PostgreSQL, actual Kafka, actual Redis—not mocks that behave differently than production.
const postgres = await new PostgreSqlContainer().start();
const connection = await createConnection(postgres.getConnectionUri());3. Implement Proper Waiting
Don't use sleep(). It's either too short (flaky) or too long (slow). Use polling with timeouts:
- Wait for specific conditions, not arbitrary time
- Set reasonable timeouts (fail fast, but not too fast)
- Include meaningful error messages when timeouts occur
4. Test the Contract, Not the Implementation
Your integration tests should verify that systems communicate correctly, not how they work internally. This makes tests more stable as implementations evolve.
The Testing Pyramid for Integrations
Contract Tests (Many)
Verify that services honor their API contracts. Tools like Pact let you define expectations and verify both sides independently. Fast, focused, catch most integration issues.
Component Tests (Some)
Test your service with real dependencies but isolated from other services. Testcontainers shine here. Slower than contract tests but catch more issues.
End-to-End Tests (Few)
Full system tests with all services running. Slowest and most expensive, but catch issues nothing else does. Keep these minimal and focused on critical paths.
The goal isn't to test everything at the integration level—it's to test the right things at the right level.
Handling External Dependencies
You can't control third-party APIs. Options:
- Sandbox environments: Use vendor test environments when available
- Service virtualization: Record and replay real API responses
- Contract testing: Verify your expectations match their documentation
- Failure injection: Test how you handle their failures
CI/CD Integration
Integration tests in CI require care:
- Run them in isolated environments (no shared databases between builds)
- Parallelize carefully (avoid port conflicts, resource contention)
- Implement retry logic for genuinely flaky external dependencies
- Track test duration and stability over time
- Quarantine flaky tests rather than ignoring failures
Measuring Success
Good integration tests have these properties:
- Deterministic: Same code produces same result, every time
- Independent: Tests don't affect each other
- Fast enough: Minutes, not hours
- Meaningful: Failures indicate real problems
- Maintained: Tests are updated when systems change
Integration testing isn't glamorous, but it's the difference between confidence and anxiety when deploying to production. Invest the time to get it right.