Update MSSQL doc for version requirements #10025

mmahacek · 2024-05-30T16:29:40Z

Update Microsoft SQL Server docs

This is an update to the MSSQL integration docs to indicate that the transaction_logs metrics requires version 2016 or later. Per MS documentation the collection requires a table that was not present in earlier server versions.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Related issues

Closes #10024

mmahacek · 2024-06-05T20:24:59Z

buildkite test this

mmahacek · 2024-06-05T20:33:10Z

This is a docs-only change - no integration code has been modified in this PR.

lalit-satapathy · 2024-06-06T05:21:31Z

Hi @mmahacek,
Thanks for submitting the PR. Couple of comments.

Looks like elastic package tool might need to be run, so that it also adds the generated README file (hence build is failing)
We could move to version criteria into the Requirements section.

Hi @aliabbas-elastic,
Can you cross-check once, if transaction logs datastream, can not run be in version below (example 2014)

ali786XI · 2024-06-06T05:33:00Z

Hi @aliabbas-elastic,
Can you cross-check once, if transaction logs datastream, can not run be in version below (example 2014)

Sure I am checking on it

ali786XI · 2024-06-06T13:16:17Z

@mmahacek I just double checked the support of transaction_log data stream in Microsoft SQL Server 2014 as per suggestion from @lalit-satapathy. I verified that we aren't to fetch metrics from sys.dm_db_log_stats as it is supported from Microsoft SQL Server 2014. But I also got metrics from the remaining tables that are mentioned as per the README. Should we mention this specifically for sys.dm_db_log_stats as rest metrics are being fetched?

Setup

Metricbeat v8.14.0

Error faced from `sys.dm_db_log_stats`

{
  "@timestamp": "2024-06-06T13:00:42.973Z",
  "@metadata": {
    "beat": "metricbeat",
    "type": "_doc",
    "version": "8.14.0"
  },
  "event": {
    "module": "mssql",
    "duration": 45641400,
    "dataset": "mssql.transaction_log"
  },
  "metricset": {
    "name": "transaction_log",
    "period": 10000
  },
  "service": {
    "address": "sqlserver://localhost?encrypt=disable",
    "type": "mssql"
  },
  "error": {
    "message": "error scanning single result: mssql: Invalid object name 'sys.dm_db_log_stats'."
  },
  "ecs": {
    "version": "8.0.0"
  },
  "host": {
    "mac": [
      "00-09-0F-FE-00-01",
      "00-2B-67-C0-39-04",
      "A8-7E-EA-BA-3D-6C",
      "A8-7E-EA-BA-3D-6D",
      "A8-7E-EA-BA-3D-70",
      "AA-7E-EA-BA-3D-6C"
    ],
    "hostname": "clw582-1255",
    "architecture": "x86_64",
    "os": {
      "kernel": "10.0.22621.3593 (WinBuild.160101.0800)",
      "build": "22631.3593",
      "type": "windows",
      "platform": "windows",
      "version": "10.0",
      "family": "windows",
      "name": "Windows 11 Pro"
    },
    "id": "57966242-af57-459a-b2ff-3628c2093bc0",
    "ip": [
      "fe80::83e5:558c:d292:fe98",
      "169.254.156.162",
      "fe80::e9e9:a842:e8ae:5b89",
      "169.254.255.39",
      "fe80::e859:c7fe:8035:ecce",
      "169.254.123.114",
      "fe80::e1a8:88bc:c21b:f0d9",
      "169.254.225.122",
      "fe80::ee96:a4f2:15ad:ca48",
      "172.20.1.3",
      "fe80::1f1f:b8b0:4a52:a3c0",
      "169.254.59.220"
    ],
    "name": "clw582-1255"
  },
  "agent": {
    "name": "CLW582-1255",
    "type": "metricbeat",
    "version": "8.14.0",
    "ephemeral_id": "72abc934-433e-41ad-898f-f42a3ea2c753",
    "id": "244fecb9-039b-46fb-93d4-d95913a690d1"
  }
}

Documents with `transaction_log` metrics (mostly related to the `sys.databases` and `sys.dm_db_log_space_usage` tables)

Document 1

{
"@timestamp": "2024-06-06T13:00:42.973Z",
"@metadata": {
  "beat": "metricbeat",
  "type": "_doc",
  "version": "8.14.0"
},
"host": {
  "name": "clw582-1255",
  "id": "57966242-af57-459a-b2ff-3628c2093bc0",
  "ip": [
    "fe80::83e5:558c:d292:fe98",
    "169.254.156.162",
    "fe80::e9e9:a842:e8ae:5b89",
    "169.254.255.39",
    "fe80::e859:c7fe:8035:ecce",
    "169.254.123.114",
    "fe80::e1a8:88bc:c21b:f0d9",
    "169.254.225.122",
    "fe80::ee96:a4f2:15ad:ca48",
    "172.20.1.3",
    "fe80::1f1f:b8b0:4a52:a3c0",
    "169.254.59.220"
  ],
  "mac": [
    "00-09-0F-FE-00-01",
    "00-2B-67-C0-39-04",
    "A8-7E-EA-BA-3D-6C",
    "A8-7E-EA-BA-3D-6D",
    "A8-7E-EA-BA-3D-70",
    "AA-7E-EA-BA-3D-6C"
  ],
  "hostname": "clw582-1255",
  "architecture": "x86_64",
  "os": {
    "type": "windows",
    "platform": "windows",
    "version": "10.0",
    "family": "windows",
    "name": "Windows 11 Pro",
    "kernel": "10.0.22621.3593 (WinBuild.160101.0800)",
    "build": "22631.3593"
  }
},
"agent": {
  "ephemeral_id": "72abc934-433e-41ad-898f-f42a3ea2c753",
  "id": "244fecb9-039b-46fb-93d4-d95913a690d1",
  "name": "CLW582-1255",
  "type": "metricbeat",
  "version": "8.14.0"
},
"event": {
  "duration": 2612160700,
  "dataset": "mssql.transaction_log",
  "module": "mssql"
},
"metricset": {
  "name": "transaction_log",
  "period": 10000
},
"mssql": {
  "transaction_log": {
    "space_usage": {
      "used": {
        "bytes": 397312,
        "pct": 51.05263137817383
      },
      "since_last_backup": {
        "bytes": 86016
      },
      "total": {
        "bytes": 778240
      }
    }
  },
  "database": {
    "name": "msdb",
    "id": 4
  }
},
"service": {
  "address": "sqlserver://localhost?encrypt=disable",
  "type": "mssql"
},
"ecs": {
  "version": "8.0.0"
}
}

Document 2

{
"@timestamp": "2024-06-06T13:00:42.973Z",
"@metadata": {
  "beat": "metricbeat",
  "type": "_doc",
  "version": "8.14.0"
},
"metricset": {
  "period": 10000,
  "name": "transaction_log"
},
"service": {
  "address": "sqlserver://localhost?encrypt=disable",
  "type": "mssql"
},
"mssql": {
  "transaction_log": {
    "space_usage": {
      "used": {
        "bytes": 466944,
        "pct": 90.47618865966797
      },
      "since_last_backup": {
        "bytes": 73728
      },
      "total": {
        "bytes": 516096
      }
    }
  },
  "database": {
    "id": 3,
    "name": "model"
  }
},
"agent": {
  "version": "8.14.0",
  "ephemeral_id": "72abc934-433e-41ad-898f-f42a3ea2c753",
  "id": "244fecb9-039b-46fb-93d4-d95913a690d1",
  "name": "CLW582-1255",
  "type": "metricbeat"
},
"ecs": {
  "version": "8.0.0"
},
"host": {
  "mac": [
    "00-09-0F-FE-00-01",
    "00-2B-67-C0-39-04",
    "A8-7E-EA-BA-3D-6C",
    "A8-7E-EA-BA-3D-6D",
    "A8-7E-EA-BA-3D-70",
    "AA-7E-EA-BA-3D-6C"
  ],
  "hostname": "clw582-1255",
  "architecture": "x86_64",
  "os": {
    "version": "10.0",
    "family": "windows",
    "name": "Windows 11 Pro",
    "kernel": "10.0.22621.3593 (WinBuild.160101.0800)",
    "build": "22631.3593",
    "type": "windows",
    "platform": "windows"
  },
  "name": "clw582-1255",
  "id": "57966242-af57-459a-b2ff-3628c2093bc0",
  "ip": [
    "fe80::83e5:558c:d292:fe98",
    "169.254.156.162",
    "fe80::e9e9:a842:e8ae:5b89",
    "169.254.255.39",
    "fe80::e859:c7fe:8035:ecce",
    "169.254.123.114",
    "fe80::e1a8:88bc:c21b:f0d9",
    "169.254.225.122",
    "fe80::ee96:a4f2:15ad:ca48",
    "172.20.1.3",
    "fe80::1f1f:b8b0:4a52:a3c0",
    "169.254.59.220"
  ]
},
"event": {
  "dataset": "mssql.transaction_log",
  "module": "mssql",
  "duration": 2610902200
}
}

Document 3

{
"@timestamp": "2024-06-06T13:00:42.973Z",
"@metadata": {
  "beat": "metricbeat",
  "type": "_doc",
  "version": "8.14.0"
},
"event": {
  "duration": 2609179900,
  "dataset": "mssql.transaction_log",
  "module": "mssql"
},
"metricset": {
  "name": "transaction_log",
  "period": 10000
},
"mssql": {
  "database": {
    "name": "tempdb",
    "id": 2
  },
  "transaction_log": {
    "space_usage": {
      "total": {
        "bytes": 516096
      },
      "used": {
        "bytes": 344064,
        "pct": 66.66666412353516
      },
      "since_last_backup": {
        "bytes": 102400
      }
    }
  }
},
"ecs": {
  "version": "8.0.0"
},
"host": {
  "ip": [
    "fe80::83e5:558c:d292:fe98",
    "169.254.156.162",
    "fe80::e9e9:a842:e8ae:5b89",
    "169.254.255.39",
    "fe80::e859:c7fe:8035:ecce",
    "169.254.123.114",
    "fe80::e1a8:88bc:c21b:f0d9",
    "169.254.225.122",
    "fe80::ee96:a4f2:15ad:ca48",
    "172.20.1.3",
    "fe80::1f1f:b8b0:4a52:a3c0",
    "169.254.59.220"
  ],
  "mac": [
    "00-09-0F-FE-00-01",
    "00-2B-67-C0-39-04",
    "A8-7E-EA-BA-3D-6C",
    "A8-7E-EA-BA-3D-6D",
    "A8-7E-EA-BA-3D-70",
    "AA-7E-EA-BA-3D-6C"
  ],
  "name": "clw582-1255",
  "hostname": "clw582-1255",
  "architecture": "x86_64",
  "os": {
    "family": "windows",
    "name": "Windows 11 Pro",
    "kernel": "10.0.22621.3593 (WinBuild.160101.0800)",
    "build": "22631.3593",
    "type": "windows",
    "platform": "windows",
    "version": "10.0"
  },
  "id": "57966242-af57-459a-b2ff-3628c2093bc0"
},
"agent": {
  "version": "8.14.0",
  "ephemeral_id": "72abc934-433e-41ad-898f-f42a3ea2c753",
  "id": "244fecb9-039b-46fb-93d4-d95913a690d1",
  "name": "CLW582-1255",
  "type": "metricbeat"
},
"service": {
  "address": "sqlserver://localhost?encrypt=disable",
  "type": "mssql"
}
}

Document 4

{
"@timestamp": "2024-06-06T13:00:42.973Z",
"@metadata": {
  "beat": "metricbeat",
  "type": "_doc",
  "version": "8.14.0"
},
"ecs": {
  "version": "8.0.0"
},
"host": {
  "name": "clw582-1255",
  "ip": [
    "fe80::83e5:558c:d292:fe98",
    "169.254.156.162",
    "fe80::e9e9:a842:e8ae:5b89",
    "169.254.255.39",
    "fe80::e859:c7fe:8035:ecce",
    "169.254.123.114",
    "fe80::e1a8:88bc:c21b:f0d9",
    "169.254.225.122",
    "fe80::ee96:a4f2:15ad:ca48",
    "172.20.1.3",
    "fe80::1f1f:b8b0:4a52:a3c0",
    "169.254.59.220"
  ],
  "mac": [
    "00-09-0F-FE-00-01",
    "00-2B-67-C0-39-04",
    "A8-7E-EA-BA-3D-6C",
    "A8-7E-EA-BA-3D-6D",
    "A8-7E-EA-BA-3D-70",
    "AA-7E-EA-BA-3D-6C"
  ],
  "hostname": "clw582-1255",
  "architecture": "x86_64",
  "os": {
    "version": "10.0",
    "family": "windows",
    "name": "Windows 11 Pro",
    "kernel": "10.0.22621.3593 (WinBuild.160101.0800)",
    "build": "22631.3593",
    "type": "windows",
    "platform": "windows"
  },
  "id": "57966242-af57-459a-b2ff-3628c2093bc0"
},
"agent": {
  "ephemeral_id": "72abc934-433e-41ad-898f-f42a3ea2c753",
  "id": "244fecb9-039b-46fb-93d4-d95913a690d1",
  "name": "CLW582-1255",
  "type": "metricbeat",
  "version": "8.14.0"
},
"service": {
  "address": "sqlserver://localhost?encrypt=disable",
  "type": "mssql"
},
"event": {
  "module": "mssql",
  "duration": 45641400,
  "dataset": "mssql.transaction_log"
},
"metricset": {
  "name": "transaction_log",
  "period": 10000
},
"mssql": {
  "database": {
    "id": 1,
    "name": "master"
  },
  "transaction_log": {
    "space_usage": {
      "since_last_backup": {
        "bytes": 233472
      },
      "total": {
        "bytes": 2088960
      },
      "used": {
        "bytes": 700416,
        "pct": 33.52941131591797
      }
    }
  }
}
}

Move version note to be more specific about which metrics are limited.

mmahacek · 2024-06-07T17:25:31Z

I moved the version note to be next to the specific query, instead of referencing limits on the entire transaction_log collection set.

packages/microsoft_sqlserver/changelog.yml

packages/microsoft_sqlserver/docs/README.md

ali786XI · 2024-06-12T05:26:20Z

@mmahacek I've updated the README and manifest. Please have a look if it looks good to you.

ali786XI

LGTM

elasticmachine · 2024-06-12T05:29:15Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

packages/microsoft_sqlserver/_dev/build/docs/README.md

elasticmachine · 2024-06-12T07:38:42Z

💚 Build Succeeded

Buildkite Build
Commit: d0f3a3d

History

💚 Build #12484 succeeded fce0d49
💔 Build #12369 failed 58cabe9
💔 Build #12281 failed 39ac40c
💔 Build #12110 failed 39ac40c

cc @mmahacek

elastic-sonarqube · 2024-06-12T07:38:44Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

elasticmachine · 2024-06-13T05:55:07Z

Package microsoft_sqlserver - 2.5.1 containing this change is available at https://epr.elastic.co/search?package=microsoft_sqlserver

Update MSSQL doc for version requirements

83b340e

mmahacek added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:microsoft_sqlserver Microsoft SQL Server labels May 30, 2024

mmahacek requested a review from a team as a code owner May 30, 2024 16:29

Update changelog.yml for MSSQL integration

39ac40c

Move note on version limit

58cabe9

Move version note to be more specific about which metrics are limited.

ali786XI reviewed Jun 8, 2024

View reviewed changes

packages/microsoft_sqlserver/changelog.yml Show resolved Hide resolved

packages/microsoft_sqlserver/docs/README.md Outdated Show resolved Hide resolved

update version and README

fce0d49

ali786XI approved these changes Jun 12, 2024

View reviewed changes

shmsr reviewed Jun 12, 2024

View reviewed changes

packages/microsoft_sqlserver/_dev/build/docs/README.md Outdated Show resolved Hide resolved

shmsr assigned mmahacek Jun 12, 2024

update version

d0f3a3d

ali786XI requested a review from shmsr June 12, 2024 07:22

shmsr approved these changes Jun 12, 2024

View reviewed changes

ali786XI merged commit f25ee85 into main Jun 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update MSSQL doc for version requirements #10025

Update MSSQL doc for version requirements #10025

Uh oh!

mmahacek commented May 30, 2024 •

edited by ali786XI

Loading

Uh oh!

mmahacek commented Jun 5, 2024

Uh oh!

mmahacek commented Jun 5, 2024

Uh oh!

lalit-satapathy commented Jun 6, 2024 •

edited

Loading

Uh oh!

ali786XI commented Jun 6, 2024

Uh oh!

ali786XI commented Jun 6, 2024 •

edited

Loading

Uh oh!

mmahacek commented Jun 7, 2024

Uh oh!

Uh oh!

Uh oh!

ali786XI commented Jun 12, 2024

Uh oh!

ali786XI left a comment

Uh oh!

elasticmachine commented Jun 12, 2024

Uh oh!

Uh oh!

elasticmachine commented Jun 12, 2024

Uh oh!

elastic-sonarqube bot commented Jun 12, 2024

Uh oh!

elasticmachine commented Jun 13, 2024

Uh oh!

Uh oh!

Update MSSQL doc for version requirements #10025

Update MSSQL doc for version requirements #10025

Uh oh!

Conversation

mmahacek commented May 30, 2024 • edited by ali786XI Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Update Microsoft SQL Server docs

Checklist

Related issues

Uh oh!

mmahacek commented Jun 5, 2024

Uh oh!

mmahacek commented Jun 5, 2024

Uh oh!

lalit-satapathy commented Jun 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ali786XI commented Jun 6, 2024

Uh oh!

ali786XI commented Jun 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Setup

Error faced from sys.dm_db_log_stats

Documents with transaction_log metrics (mostly related to the sys.databases and sys.dm_db_log_space_usage tables)

Uh oh!

mmahacek commented Jun 7, 2024

Uh oh!

Uh oh!

Uh oh!

ali786XI commented Jun 12, 2024

Uh oh!

ali786XI left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 12, 2024

🚀 Benchmarks report

Uh oh!

Uh oh!

elasticmachine commented Jun 12, 2024

💚 Build Succeeded

History

Uh oh!

elastic-sonarqube bot commented Jun 12, 2024

Quality Gate passed

Uh oh!

elasticmachine commented Jun 13, 2024

Uh oh!

Uh oh!

mmahacek commented May 30, 2024 •

edited by ali786XI

Loading

lalit-satapathy commented Jun 6, 2024 •

edited

Loading

ali786XI commented Jun 6, 2024 •

edited

Loading

Error faced from `sys.dm_db_log_stats`

Documents with `transaction_log` metrics (mostly related to the `sys.databases` and `sys.dm_db_log_space_usage` tables)