Skip to content

Commit 5301c85

Browse files
muskan-agarwal26flexitrev
muskan-agarwal26
authored and
flexitrev
committed
[proofpoint_itm] Initial release of Proofpoint ITM (#13153)
Add report data stream including collection and ingest logic with mapping to the ECS, and dashboard visualisations. Pipeline and system test data have been extracted from Splunk and sanitized to ensure data integrity.
1 parent 4c16fc8 commit 5301c85

23 files changed

+3563
-0
lines changed

‎.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -335,6 +335,7 @@
335335
/packages/prometheus/data_stream/collector @elastic/obs-infraobs-integrations
336336
/packages/prometheus/data_stream/query @elastic/obs-infraobs-integrations
337337
/packages/prometheus_input @elastic/obs-infraobs-integrations
338+
/packages/proofpoint_itm @elastic/security-service-integrations
338339
/packages/proofpoint_on_demand @elastic/security-service-integrations
339340
/packages/proofpoint_tap @elastic/security-service-integrations
340341
/packages/proxysg @elastic/sec-deployment-and-devices

‎packages/proofpoint_itm/_dev/build/build.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
dependencies:
2+
ecs:
3+
reference: "git@v8.17.0"

‎packages/proofpoint_itm/_dev/build/docs/README.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Proofpoint Insider Threat Management (ITM)
2+
3+
[Proofpoint Insider Threat Management (ITM)](https://www.proofpoint.com/us/products/insider-threat-management) is a people-centric SaaS solution that helps you protect sensitive data from insider threats and data loss at the endpoint. It combines context across content, behavior and threats to provide you with deep visibility into user activities. Proofpoint ITM helps security teams tackle the challenges of detecting and preventing insider threats. It can streamline their responses to insider-led incidents and provide insights that help prevent further damage.
4+
5+
Use this integration to collect and parse data from your Proofpoint ITM instance.
6+
7+
## Compatibility
8+
9+
This module has been tested against the Proofpoint ITM API version **v2**.
10+
11+
## Data streams
12+
13+
This integration collects the following logs:
14+
15+
- **Reports** - This data stream enables users to retrieve reports from Proofpoint ITM, encompassing the below log types:
16+
1. User activity
17+
2. DBA activity
18+
3. System events
19+
4. Alerts activity
20+
5. Audit activity
21+
6. In-App elements
22+
23+
## Requirements
24+
25+
### Agentless Enabled Integration
26+
27+
Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and the [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html).
28+
29+
Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.
30+
31+
### Agent Based Installation
32+
33+
- Elastic Agent must be installed
34+
- You can install only one Elastic Agent per host.
35+
- Elastic Agent is required to stream data from the GCP Pub/Sub or REST API and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines.
36+
37+
#### Installing and managing an Elastic Agent:
38+
39+
You have a few options for installing and managing an Elastic Agent:
40+
41+
#### Install a Fleet-managed Elastic Agent (recommended):
42+
43+
With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.
44+
45+
#### Install Elastic Agent in standalone mode (advanced users):
46+
47+
With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.
48+
49+
#### Install Elastic Agent in a containerized environment:
50+
51+
You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry and we provide deployment manifests for running on Kubernetes.
52+
53+
There are some minimum requirements for running Elastic Agent and for more information, refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
54+
55+
## Setup
56+
57+
Follow the [ITM On-Prem (ObserveIT) API Portal](https://prod.docs.oit.proofpoint.com/configuration_guide/observeit_api_portal.htm) guide to setup the Proofpoint ITM On-Prem API Portal.
58+
59+
### Enabling the integration in Elastic:
60+
61+
1. In Kibana navigate to Management > Integrations.
62+
2. In "Search for integrations" top bar, search for `Proofpoint ITM`.
63+
3. Select the "Proofpoint ITM" integration from the search results.
64+
4. Select "Add Proofpoint ITM" to add the integration.
65+
5. Add all the required integration configuration parameters, including the URL, Token URL, Client ID, and Client type, to enable data collection.
66+
6. Select "Save and continue" to save the integration.
67+
68+
## Logs reference
69+
70+
### Report
71+
72+
This is the `report` dataset.
73+
74+
#### Example
75+
76+
{{event "report"}}
77+
78+
{{fields "report"}}

‎packages/proofpoint_itm/_dev/deploy/docker/docker-compose.yml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
version: '3.0'
2+
services:
3+
proofpoint_itm:
4+
image: docker.elastic.co/observability/stream:v0.17.1
5+
hostname: proofpoint_itm
6+
ports:
7+
- 8090
8+
volumes:
9+
- ./files:/files:ro
10+
environment:
11+
PORT: '8090'
12+
command:
13+
- http-server
14+
- --addr=:8090
15+
- --config=/files/config.yml

‎packages/proofpoint_itm/_dev/deploy/docker/files/config.yml

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
rules:
2+
- path: /v2/apis/auth/oauth/token
3+
methods: ['POST']
4+
request_headers:
5+
Content-Type:
6+
- 'application/x-www-form-urlencoded'
7+
responses:
8+
- status_code: 200
9+
headers:
10+
Content-Type:
11+
- 'application/json'
12+
body: |
13+
{"access_token":"xxxx","expires_in":3600,"token_type":"Bearer"}
14+
- path: /v2/apis/report;realm=observeit/reports
15+
methods: ['GET']
16+
query_params:
17+
since: "{since:20.*}"
18+
request_headers:
19+
Authorization:
20+
- 'Bearer xxxx'
21+
responses:
22+
- status_code: 200
23+
body: |-
24+
{{ minify_json `
25+
{
26+
"data": [
27+
{
28+
"_time": "2025-02-25T16:08:11Z",
29+
"accessedSiteName": "c-awfi.top",
30+
"accessedUrl": "http://c-awfi.top/",
31+
"applicationName": "Windows Shell Experience Host",
32+
"collectorId": "C2C1C429-C002-4FB8-99F4-7F1005ED9889",
33+
"collectorUrl": "https://code1.preview.observeit.net/",
34+
"command": "example_command",
35+
"commandParams": "--example --params",
36+
"createdAt": "2025-02-25T16:08:11Z",
37+
"domainName": "code1.observeit.net",
38+
"endpointId": "E035BBC2-1D72-4F25-9408-2BD807FB7B13",
39+
"endpointName": "Example Endpoint",
40+
"host": "host.example.com",
41+
"id": "7340EB6D-A8BB-4F25-9408-2BD807FB7B13",
42+
"loginName": "Administrator",
43+
"observedAt": "2025-02-25T16:08:11Z",
44+
"os": "Windows",
45+
"playbackUrl": "https://code1.preview.observeit.net/ObserveIT/SlideViewer.aspx?SessionID=1A8B5249-EDAC-A8BB-4F25-9408-2BD807FB7B13",
46+
"processExecutable": "shellexexperiencehost",
47+
"remoteAddress": "175.16.199.0",
48+
"remoteHostName": "Dake-WinX",
49+
"risingValue": "2025-03-01T12:00:00Z",
50+
"secondaryDomainName": "n/a",
51+
"secondaryLoginName": "n/a",
52+
"sessionUrl": "https://session.example.com/abc123",
53+
"timezoneOffset": "0",
54+
"windowTitle": "Start",
55+
"sessionId": "1A8B52A9-EDAC-448E-9871-79DB21D53C28",
56+
"databaseName": "example_database",
57+
"details": "Detailed description of the event.",
58+
"detailsUrl": "https://details.example.com/event/abcde",
59+
"eventPlaybackUrl": "https://playback.example.com/event/abcde",
60+
"ruleCategoryName": "Security",
61+
"ruleDesc": "Description of the security rule.",
62+
"ruleName": "Invalid User Asstempt",
63+
"severity": "High",
64+
"sqlCommand": "SELECT * FROM users;",
65+
"sqlUserName": "db_user",
66+
"userActivityEventId": 9876543210,
67+
"userActivityObservedAt": "2025-02-25T16:08:11Z",
68+
"operationKind": "Read",
69+
"originFileName": "confidential.docx",
70+
"originSiteName": "Internal SharePoint",
71+
"targetFileName": "confidential_copy.docx",
72+
"targetSiteName": "External Drive"
73+
},
74+
{
75+
"_time": "2125-02-25T16:09:11Z",
76+
"accessedSiteName": "c-awfi.top",
77+
"accessedUrl": "http://c-awfi.top/",
78+
"applicationName": "Windows Shell Experience Host",
79+
"collectorId": "C2C1C429-C002-4FB8-99F4-7F1005ED9889",
80+
"collectorUrl": "https://code1.preview.observeit.net/",
81+
"command": "example_command",
82+
"commandParams": "--example --params",
83+
"createdAt": "2125-02-25T16:08:11Z",
84+
"domainName": "code1.observeit.net",
85+
"endpointId": "E035BBC2-1D72-4F25-9408-2BD807FB7B13",
86+
"endpointName": "Example Endpoint",
87+
"host": "host.example.com",
88+
"id": "7340EB6D-A8BB-4F25-9408-2BD807FB7B13",
89+
"loginName": "Administrator",
90+
"observedAt": "2125-02-25T16:08:11Z",
91+
"os": "Windows",
92+
"playbackUrl": "https://code1.preview.observeit.net/ObserveIT/SlideViewer.aspx?SessionID=1A8B5249-EDAC-A8BB-4F25-9408-2BD807FB7B13",
93+
"processExecutable": "shellexexperiencehost",
94+
"remoteAddress": "175.16.199.0",
95+
"remoteHostName": "Dake-WinX",
96+
"risingValue": "2125-03-01T12:00:00Z",
97+
"secondaryDomainName": "n/a",
98+
"secondaryLoginName": "n/a",
99+
"sessionUrl": "https://session.example.com/abc123",
100+
"timezoneOffset": "0",
101+
"windowTitle": "Start",
102+
"sessionId": "1A8B52A9-EDAC-448E-9871-79DB21D53C28",
103+
"databaseName": "example_database",
104+
"details": "Detailed description of the event.",
105+
"detailsUrl": "https://details.example.com/event/abcde",
106+
"eventPlaybackUrl": "https://playback.example.com/event/abcde",
107+
"ruleCategoryName": "Security",
108+
"ruleDesc": "Description of the security rule.",
109+
"ruleName": "Invalid User Asstempt",
110+
"severity": "High",
111+
"sqlCommand": "SELECT * FROM users;",
112+
"sqlUserName": "db_user",
113+
"userActivityEventId": 9876543210,
114+
"userActivityObservedAt": "2125-02-25T16:08:11Z",
115+
"operationKind": "Read",
116+
"originFileName": "confidential.docx",
117+
"originSiteName": "Internal SharePoint",
118+
"targetFileName": "confidential_copy.docx",
119+
"targetSiteName": "External Drive"
120+
}
121+
]
122+
}
123+
`}}
124+
- path: /v2/apis/report;realm=observeit/reports
125+
methods: ['GET']
126+
query_params:
127+
since: "{since:21.*}"
128+
request_headers:
129+
Authorization:
130+
- 'Bearer xxxx'
131+
responses:
132+
- status_code: 200
133+
body: |-
134+
{{ minify_json `
135+
{
136+
"data": [
137+
{
138+
"_time": "2125-02-25T16:08:11Z",
139+
"accessedSiteName": "c-awfi.top",
140+
"accessedUrl": "http://c-awfi.top/",
141+
"applicationName": "Windows Shell Experience Host",
142+
"collectorId": "C2C1C429-C002-4FB8-99F4-7F1005ED9889",
143+
"collectorUrl": "https://code1.preview.observeit.net/",
144+
"command": "example_command",
145+
"commandParams": "--example --params",
146+
"createdAt": "2125-02-25T16:08:11Z",
147+
"domainName": "code1.observeit.net",
148+
"endpointId": "E035BBC2-1D72-4F25-9408-2BD807FB7B13",
149+
"endpointName": "Example Endpoint",
150+
"host": "host.example.com",
151+
"id": "7340EB6D-A8BB-4F25-9408-2BD807FB7B13",
152+
"loginName": "Administrator",
153+
"observedAt": "2125-02-25T16:08:11Z",
154+
"os": "Windows",
155+
"playbackUrl": "https://code1.preview.observeit.net/ObserveIT/SlideViewer.aspx?SessionID=1A8B5249-EDAC-A8BB-4F25-9408-2BD807FB7B13",
156+
"processExecutable": "shellexexperiencehost",
157+
"remoteAddress": "175.16.199.0",
158+
"remoteHostName": "Dake-WinX",
159+
"risingValue": "2225-03-01T12:00:00Z",
160+
"secondaryDomainName": "n/a",
161+
"secondaryLoginName": "n/a",
162+
"sessionUrl": "https://session.example.com/abc123",
163+
"timezoneOffset": "0",
164+
"windowTitle": "Start",
165+
"sessionId": "1A8B52A9-EDAC-448E-9871-79DB21D53C28",
166+
"databaseName": "example_database",
167+
"details": "Detailed description of the event.",
168+
"detailsUrl": "https://details.example.com/event/abcde",
169+
"eventPlaybackUrl": "https://playback.example.com/event/abcde",
170+
"ruleCategoryName": "Security",
171+
"ruleDesc": "Description of the security rule.",
172+
"ruleName": "Invalid User Asstempt",
173+
"severity": "High",
174+
"sqlCommand": "SELECT * FROM users;",
175+
"sqlUserName": "db_user",
176+
"userActivityEventId": 9876543210,
177+
"userActivityObservedAt": "2125-02-25T16:08:11Z",
178+
"operationKind": "Read",
179+
"originFileName": "confidential.docx",
180+
"originSiteName": "Internal SharePoint",
181+
"targetFileName": "confidential_copy.docx",
182+
"targetSiteName": "External Drive"
183+
}
184+
]
185+
}
186+
`}}

‎packages/proofpoint_itm/changelog.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# newer versions go on top
2+
- version: "0.1.0"
3+
changes:
4+
- description: Initial release.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/13153

‎packages/proofpoint_itm/data_stream/report/_dev/test/pipeline/test-alert.log

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"_time":"2025-02-25T16:08:11Z","accessedSiteName":"Example Site","accessedUrl":"https://www.example.com","applicationName":"Windows Shell Experience Host","collectorId":"C2C1C429-C002-4FB8-99F4-7F1005ED9889","collectorUrl":"https://code1.preview.observeit.net/","command":"example_command","commandParams":"--example --params","createdAt":"2025-02-25T16:08:11Z","domainName":"code1.observeit.net","endpointId":"E035BBC2-1D72-4F25-9408-2BD807FB7B13","endpointName":"Example Endpoint","host":"host.example.com","id":"7330EB6D-A8BB-4F25-9408-2BD807FB7B13","loginName":"Administrator","observedAt":"2025-02-25T16:08:11Z","os":"Windows","playbackUrl":"https://code1.preview.observeit.net/ObserveIT/SlideViewer.aspx?SessionID=1A8B5249-EDAC-A8BB-4F25-9408-2BD807FB7B13","processExecutable":"shellexexperiencehost","remoteAddress":"192.168.1.1","remoteHostName":"Michaels-MacBoo","risingValue":"2018-06-06T17:43:18.446Z","secondaryDomainName":"n/a","secondaryLoginName":"n/a","sessionUrl":"https://session.example.com/abc123","timezoneOffset":"0","windowTitle":"Start","sessionId":"1A8B52A9-EDAC-448E-9871-79DB21D53C28","databaseName":"example_database","details":"Detailed description of the event.","detailsUrl":"https://details.example.com/event/abcde","eventPlaybackUrl":"https://playback.example.com/event/abcde","ruleCategoryName":"Security","ruleDesc":"Description of the security rule.","ruleName":"Unauthorized Access Attempt","severity":"High","sqlCommand":"SELECT * FROM users;","sqlUserName":"db_user","userActivityEventId":9876543210,"userActivityObservedAt":"2025-02-25T16:08:11Z","operationKind":"Read","originFileName":"confidential.docx","originSiteName":"Internal SharePoint","targetFileName":"confidential_copy.docx","targetSiteName":"External Drive"}

0 commit comments

Comments
 (0)