[github] Add dependabot alerts data stream

[kcreddy]

What does this PR do?

For ingesting Github Advanced Security (GHAS) Dependabot alerts

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Screenshots

Demo

Demo of Github Advanced Security (GHAS) Alerts:

GHAS.Demo.mp4

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-08-24T11:39:28.359+0000

• Duration: 17 min 56 sec

Test stats 🧪

Test	Results
Failed	0
Passed	49
Skipped	0
Total	49

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (4/4)	💚
Files	100.0% (4/4)	💚 2.825
Classes	100.0% (4/4)	💚 2.825
Methods	100.0% (45/45)	💚 10.619
Lines	94.958% (678/714)	👍 3.961
Conditionals	100.0% (0/0)	💚

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

Could we store this as a formatted string to make reading it easier? Like

  query: |-
    query fetchRepoAlerts($org: String!, $repo: String!) {
      repository(owner: $org, name: $repo) {
        vulnerabilityAlerts(first: 2) {
          nodes {
            createdAt
            dependabotUpdate {
              error {
                body
                errorType
                title
              }
              pullRequest {
                createdAt
                closed
                closedAt
                merged
                mergedAt
                number
                url
                title
              }
            }
            dependencyScope
            dismissReason
            dismissedAt
            dismisser {
              login
              url
            }
            fixReason
            fixedAt
            number
            repository {
              description
              isInOrganization
              isPrivate
              name
              owner {
                login
                url
              }
              url
            }
            securityAdvisory {
              classification
              cvss {
                score
                vectorString
              }
              cwes(first: 100) {
                nodes {
                  cweId
                  description
                  name
                }
              }
              description
              ghsaId
              identifiers {
                type
                value
              }
              origin
              permalink
              references {
                url
              }
              publishedAt
              severity
              summary
              updatedAt
              withdrawnAt
            }
            securityVulnerability {
              firstPatchedVersion {
                identifier
              }
              package {
                ecosystem
                name
              }
              severity
              updatedAt
              vulnerableVersionRange
            }
            state
            vulnerableManifestPath
            vulnerableManifestFilename
            vulnerableRequirements
          }
          pageInfo {
            hasNextPage
            endCursor
          }
        }
      }
    }

[kcreddy]

@andrewkroh Thanks for the suggestion. Unfortunately, the above formatted string doesn't seem to be working. Following is the error message in the agent logs:
"message":"error processing response: template: :1:69: executing \"\" at <.last_response.body.data.repository.vulnerabilityAlerts.pageInfo.endCursor>: map has no entry for key \"data\""
I think the request might be getting an empty response and trying to get cursor info where there is none and logging it.

[P1llus]

I don't think the request body gets marshalled correctly that way, a bit unsure if that is just because of a typo or not

[andrewkroh]

Would it be possible to construct a query that gets the alerts from all repos in an org? As repos are added within an org, then you will need to keep adding new inputs which isn't sustainable for big orgs.

[andrewkroh]

This is an incomplete PoC, but it appears that it is possible (I know very little about graphql). So perhaps the integration could function is two modes like org mode or single repo mode. I think org mode will be used the most.

query orgRepoVulnerabilities($org: String!) {
  organization(login: $org) {
    repositories(first: 100) {
      nodes {
        name
        vulnerabilityAlerts(first: 10) {
          nodes {
            createdAt
            dependabotUpdate {
              error {
                body
                errorType
                title
              }
              pullRequest {
                createdAt
                closed
                closedAt
                merged
                mergedAt
                number
                url
                title
              }
            }
          }
        }
      }
      pageInfo {
        hasNextPage
        endCursor
      }
    }
  }
}

[kcreddy]

This is a great suggestion @andrewkroh. When I met with Github folks they also wanted organisation level alerts implemented. Here is the PR where this enhancement will be implemented(targeting for 8.5): #3935

The reason repository was targeted in the first run was to have some consistency in the implementation as their REST APIs for Github Code Scanning alerts at ORG level wasn't working. So, thought of adding repo endpoint first, then followup with org endpoint and provide choice to the user.

[P1llus]

Yeah the end goal is to offer 2 options.
Organization level as default, and a repo option in advanced options, if repo is set, it will focus on a single repo, as there is no way to filter on repo's at all when using organization/root level API's.

[kcreddy]

Code changed to query organization level alerts by default, and if repository (optional) is provided, the integration would only query repository level alerts.

[efd6]

The \u encoding of < is unfortunate. This is also present in event.original which seems worse, verging on a bug.

ISTM that this should be fixed in httpjson (https://play.golang.com/p/YGhWbtX7jO0)

[P1llus]

LGTM

[kcreddy]

Demo of Github Advanced Security (GHAS) Alerts:

GHAS.Demo.mp4

[andrewkroh]

Great demo! Thank you for adding that!

Feedback

• Consider moving the Personal Access Token config into main manifest so that only needs to be configured once. As an example, we do this in the aws integration for the access / secret tokens.

• I'm not sure if there is precedent, but should the Personal Access Token be marked as a password type to mask it in the UI?

• Since all of the individual visualizations in the dashboards are supposed to be embedded in the dashboard (as opposed to stored as an independent saved object), you no longer need to title the visualizations with the suffix of the dashboard name. That is, you can remove the [Githbub Code Scanning] from viz titles.

[kcreddy]

Thanks for the feedback @andrewkroh! I will be implementing above suggestions in the subsequent release.