API Insights is an open-source tool that helps developers improve API quality and security.

  • By Cisco Developer
  • Last update: Jan 3, 2023
  • Comments: 10

Go Report Card published

API Insights

API Insights Logo

API Insights is a tool to enable organizations to manage versioned API specifications (Swagger 2.0/OpenAPI Spec 3.x) for services. It also does static analysis of API spec files for compliance against REST API best practices guidelines, document completeness, inclusive language check and runtime API drift from documented spec. To help API consumers and developers, API Insights service also supports generating an API changelog including identification of backward compatibility breaking changes between 2 versions of API spec files.

API Specifications Challenges

  • As the number of services increases, no common place for storing versioned API specs.
  • Inconsistency in API specifications across teams. Makes it difficult for API consumers that integrate across multiple APIs.
  • API changes across versions could result in breaking backward compatibility.
  • Lack of consistent documentation of API changes across multiple releases.

Solution

API.Insights-Demo.Video.mp4
  • API Insights service enables storing of multiple versions of released (& release candidate) API specifications.
  • Validate & Score API Spec against guidelines:
  • API spec diff across multiple versions/revisions
    • Identify and alert on backward compatibility breaking changes.
    • API Insights CLI to enable running spec analyzer as a part of API spec CI/CD or local commit pipeline.

User Flow and Architecture

API Insights

User Flow

  • Developer or Tech Lead can upload the API Specification and subsequent revisions:
    • Commit new version/revision of spec in GitHub repository.
    • CI/CD pipeline with specs analysis against guidelines & generate report/score.
    • On GitHub release tag, new version/revision of spec will be uploaded to the API Insights service by CI/CD task.
    • Multiple API specs across products/services can be managed in the API Insights service.
  • On new spec upload, preconfigured analyzers will run on spec in background.
  • User can go API Insights UI to view:
    • Analyzer score and issue listing with trends across releases.
    • Detailed report with severity, line number and remediation recommendations.
  • Users will be able to see a summary of all API changes (New, Modified, Removed & Breaking) and will be able to see the detailed spec diff by clicking on each changed item.
  • Integration with APIClarity & Panoptica will allow:
    • Security and Compliance users to get reports on Zombie & Shadow APIs
    • Reconstructed OAPI for missing specs
    • Security Analysis of API

Related Projects and resources

Getting Started

This repo contains a Helm based deployer that can be deployed in a local Kubernetes cluster setup using like Rancher Desktop, minikube etc. The detailed instructions are found here.

Development setup

Build and start UI & backend services using Docker Compose

docker-compose up 

Once Docker Compose is up, UI and be access at http://localhost:8080

  • To run the API service natively outside docker, refer to api/README.md
  • To run the UI natively outside docker, refer to ui/README.md

Note: Docker-compose requires installing Rancher Desktop or licensed 'Docker Desktop'.

Contribution

We welcome contributions, please find details in CONTRIBUTING.md

Download

api-insights.zip

Comments(10)

  • 1

    backend: data compression feature.

    Utilizes GORM's hooks (https://gorm.io/docs/hooks.html) to compress (before saving) & decompress (after querying) large-sized data (size threshold determined by config start-data-compression-at-bytes).

    start-data-compression-at-bytes: Value in bytes at when to compress blob; default=3145728,0=compress all,-1=no compression

    Used for Spec.Doc, SpecAnalysis.SpecAnalysisResult.RawResult, SpecDiff.SpecDiffResult.RawResult.

  • 2

    change mysql to 5.7.40 to avoid sort buffer oom

    This is to avoid sort buffer oom in mysql 8.x. Please note, If you meet some issue to docker compose up this version of mysql. It probably because the old data of mysql 8.x. You need to remove them by: docker compose down --volumes

  • 3

    Show the weight of each analyzer and its score

    In the current report page, we can only see a list of analyzers used on a spec. However, it is not clear how much weight was applied and how the score was calculated for each analyzer.

    image

    To address this issue, the UI will need some adjustments to display these information.

    image

  • 4

    Nginx body size

    increase nginix client_max_body_size so that larger specs can be uploaded successfully. Fixes: https://github.com/cisco-developer/api-insights/issues/50

  • 5

    backend: remove tmp diff files

    Remove diff files that are temporarily created during diff execution.

    These files were originally retained for testing purpose, but can now be cleaned up.

  • 6

    Minor README enhancements

    Fixed some typos, and made some enhancements for clarity & consistency.

    For example, use consistent form of CI/CD, use accurate branding terms like OpenAPI instead of Open API, GitHub instead of github, Docker Compose instead of docker-compose, etc.

  • 7

    NGINX issue while deploying in Openshift

    After deploying helm chart in Openshift platform , facing below error. Please help on resolving below error.

    sed: can't create temp file '/etc/nginx/nginx.confXXXXXX': Permission denied 2022/12/12 07:00:17 [emerg] 8#8: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied) nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)

    For this issue, Stackoverflow suggested below mentioned solution to rebuild image. I have to build and package UI application. Please let me know UI setup details.

    https://github.com/nginxinc/docker-nginx/issues/417

  • 8

    Update CLI

    • cli: update analyze cmd with flag --fail-on-error-rule (defaults to exit code 0 w/o the flag)
    • cli: prevents Cobra from creating a default 'completion' command
    • cli: prevents Cobra from generating docs with "Auto generated by spf13/cobra..."
  • 9

    Diff & breaking change reference object preview

    Scenario: When showing diff between 2 versions of the specs doc or highlighting backward compatibility breaking changes, sometimes changes are not visible in the API Path request or response object but changes are in reference objects. Reference objects are typically defined under components (OAS3) or definitions (swagger2). For enabling users to do a more meaningful analysis of spec diff, API Insights UI needs to support allowing showing diff of referred components in the context of API Path.

    Proposed Solution:

    1. UX should be improved to allow users to see the diff between a referenced object in the API Path context. For example in the sample case "#/definition/ListResponse" can be expanded to see the diff between ListResponse objects in 2 versions. diff-reference-view

    2. New API should be exposed by the API Insights service that will return reference object GET /v1/apiregistry/services/{id}/specs/{specID}/references

      • OpenAPI 3.0 : return 'components' that include schemas, parameters, responses etc. as child element.
      • Swagger2 : return definitions, parameters, responses and securityDefinitions section of specs.
        • Check https://swagger.io/docs/specification/components/ "Differences From OpenAPI 2.0" section for more details.
    3. UI while showing diff should detect "$ref": "#/components/" in the diff JSON/YAML & enable view reference functionality as per design in step 1.

      • Note: Only local reference object details viewing will be supported. Reference pointing to components in different files or remote URLs will not be supported.
  • 10

    Large API spec file upload failing.

    Issue: When uploading a large spec file for a service, upload failed with an error request entity too large. See below snapshot

    Screen Shot 2022-11-07 at 5 56 15 PM

    Expected Behavior: Upload spec should work and spec should get analyzed.