ReportPortal

Visit Website

Introduction

ReportPortal is a server that collects automated test results as the run happens. Agents for JUnit, TestNG, pytest, Cucumber, Playwright, and other runners post suites, tests, logs, and screenshots to the API. The UI shows the launch in progress, groups failures, and stores history across builds. An analyzer service suggests which failures look like ones you have already triaged.

The server is Apache-2.0. The meta-repo is reportportal/reportportal; the running system is several services (API, UI, authorization, analyzer) plus Postgres, OpenSearch or Elasticsearch, RabbitMQ, and object storage. You can run that stack yourself. ReportPortal also sells support packages and hosted options. A static HTML report such as Allure is a different shape: files in CI, not a long-lived server.

Features

1. Live launches: Results appear while CI is still running, so a long suite can be read before the job exits.
2. Agents: Framework-specific agents (agent-*) and HTTP clients (client-*) map test events onto ReportPortal items. You add the agent to the test project; you do not rewrite assertions.
3. History and flaky marking: A test that passes and fails across launches can be marked and tracked instead of treated as a new failure every time.
4. Auto-analysis: The analyzer compares a new failure with previous ones and can suggest a defect type. The suggestion is only as good as the triage data already in the project.
5. Quality gates: A launch can fail the pipeline when pass rate, or a custom rule, misses a threshold.
6. Dashboards and widgets: Pass rate, failure reasons, and duration trends are widgets on a project dashboard.
7. Bug trackers: Plugins link an item to Jira or Rally. Sauce Labs is also a published plugin.
8. Docker and Kubernetes installs: A compose file brings up a demo stack. Production installs split services and storage.

Pros

1. One database of launches: Results survive the CI agent. You can compare Tuesday’s API suite with last month’s without downloading artifacts.
2. Works across runners: Java, Python, JavaScript, and .NET agents post into the same project model.
3. Triage sticks: Once someone marks a failure as a product bug, later similar failures can be grouped instead of re-investigated from a blank log.
4. Self-host is allowed: Apache-2.0 and a Docker install mean the results stay in your network.
5. The test code barely changes: An agent is a dependency and a config file with an API token, not a new assertion style.

Cons

1. The stack is heavy: API, UI, auth, analyzer, a database, search, a queue, and object storage. A single HTML file is less work if you only need last night’s report.
2. Analysis is not ground truth: The matcher suggests defect types. A new error text still needs a person. Bad triage data trains bad suggestions.
3. Agents must match the server: A version skew between agent-java and the API shows up as missing logs, not as a clean error in the test.
4. Premium features sit on paid packages: Quality gates and SCIM are listed as premium features bundled with ReportPortal service packages, not as anonymous free-cloud defaults.
5. Operations: Upgrades touch several containers. Plan backups for Postgres and the blob store before the report becomes the system of record.

Usage and Links (Example)

A local demo follows the compose guide in the reportportal/reportportal repo. After the UI is up, create a project and an API key, then point an agent at it. For pytest:

pip install pytest-reportportal

# pytest.ini
[pytest]
rp_api_key = <uuid>
rp_endpoint = http://localhost:8080
rp_project = default_personal
rp_launch = local

Run pytest as usual. The agent posts the session to that launch. Java agents are Maven or Gradle dependencies plus a reportportal.properties file. Use the agent docs that match your server version.

Resources:

• ReportPortal website
• Documentation
• Service package pricing
• GitHub: reportportal/reportportal

Pricing

1. Community edition: Free to self-host. The project documents Docker, Kubernetes, and a Google Cloud Marketplace image. License is Apache-2.0.
2. Service packages (published on reportportal.io): Silver is $10,000 per year for 50 support hours. Gold is $20,000 per year for 150 hours. Platinum is $50,000 per year for 600 hours. Hours are for the year you paid for and do not roll over. The packages include premium features such as quality gates and SCIM, plus a 24-hour response target.
3. Hosted SaaS: ReportPortal is also offered as a hosted service. The official page above prices support time, not a simple per-seat cloud card. Third-party listings quote shared and dedicated instances at different monthly rates. Confirm the current hosted number with ReportPortal before you budget one; this entry does not treat those listings as the vendor price.

Recommended for?

1. CI that already runs several frameworks and needs one place to triage failures across days.
2. Teams with someone to operate Docker or Kubernetes, or a budget for a hosted instance and a support package.
3. Suites large enough that opening last night’s HTML is no longer enough, especially when the same failure returns for weeks.
4. Groups that will mark defect types. Auto-analysis without that discipline is just another unowned dashboard.

Conclusion

ReportPortal is a real test-results server: live launches, history, and optional failure clustering, under an open-source license if you run it. The cost is the stack, or a support package starting at $10,000 a year, not a library dependency. If a static report is enough, Allure Report is the smaller tool. If you need the server, pin agent versions to the server you actually deployed.