Definition
Analyzing system calls to determine whether a process is exhibiting unauthorized behavior.
How it works
System calls are APIs between a user application and the operating system [1].
By analyzing a process's use of these APIs, it is, in some cases, possible to ascertain whether a program is exhibiting unauthorized behavior, including trying to escalate its privileges.
Gathering System Calls
A common method to capture system calls is to use kernel APIs to hook [2] a process's system call invocations.
The Linux system call `ptrace` tracks other system calls in a process and allows their alteration; this is made use of by GDB. `strace` utilizes `ptrace` and will print to stdout each system call invoked. Other applications record this data in local or remote databases.
The log entry for each system call, which may reference additional information such as the date and time, and the process tree for the process which made the system call, is relayed, in real time or post-facto, to an analysis module which consults a catalog or model to determine whether the distribution matches a known-good or known-bad pattern.
Analysis
System calls are analyzed with a variety of methods. Some analytics look for specific sequences of instructions, others may apply statistical methods to identify abnormal behavior. Sequences of instructions can be abstracted into conceptually higher order user activities, for example:
- An attacker executes many system calls in a short period of time, with several sequences which could be used to escalate privileges.
- Getting the contents from a URL, writing to a new file, and then executing the same file.
- A ransomware program which either uses a loop or creates many threads to: read a specified file, encrypt its contents, create an output file with a similar name to the original file, and delete the unencrypted original.
Considerations
[1] [Syscalls](http://man7.org/linux/man-pages/man2/syscalls.2.html)
[2] [Hooking](http://dbpedia.org/resource/Hooking)
- Duplicative or extraneous system calls may be added to malware to defeat analytics.
- Malware could replace API hooking instructions to allow system calls to be made without being monitored.
- A model built from a training set of system calls and related data may not be updated fast enough to detect new threats.
Implementation perspective
System Call Analysis should be treated as a technical defensive capability rather than a product checkbox. In practice, teams should define the protected scope, the conditions under which the technique acts, and the observable evidence that demonstrates the intended behavior. For this technique, likely engineering context includes System Call.
Use the technique to identify suspicious, unauthorized, or abnormal activity through observable evidence and repeatable analysis.
Questions to ask
- Which events, states, or artifacts must be observed for the analysis to work?
- What analytic logic, threshold, comparison, or signature turns observations into a finding?
- How are expected false positives, blind spots, and environmental variations documented?
- Who receives the result, and what action is expected when the technique produces a finding?
Evidence and validation
- Telemetry and data-source configuration records
- Analytic logic, thresholds, signatures, and version history
- Test cases demonstrating expected positive and negative results
- Alert, triage, escalation, and tuning records
Common failure patterns
- Required telemetry is missing, delayed, or transformed in a way that invalidates the analysis.
- The technique produces alerts without an accountable triage and response process.
- Detection coverage is claimed from product deployment without testing the relevant analytic behavior.
This implementation perspective is original Bare Metal Cyber educational content. It does not replace the official D3FEND definition or establish that a specific product implements the technique.
Technique hierarchy
Top-level family
Parent techniques
Direct child techniques
Artifacts and ontology entities
These relationships describe how D3FEND connects a defensive technique to artifacts or other ontology entities. They describe graph semantics, not a product certification.
Explicit technique relationships
Show inferred artifact relationship paths (1)
Offensive-technique relationships
These relationships are generated from D3FEND graph paths and are explicitly experimental. They should be treated as hypotheses for defensive analysis—not as proof that the technique prevents, detects, or removes an offensive behavior.
ATT&CK Enterprise (40)
ATT&CK for ICS (7)
Authoritative sources
- Open this technique on the official D3FEND website ↗
- Open the official ontology resources ↗
- CAR-2020-05-001: MiniDump of LSASS ↗MITRE · MITRE · ExternalKnowledgeBase
- CAR-2021-05-011: Create Remote Thread into LSASS ↗MITRE · MITRE · ExternalKnowledgeBase
- CAR-2019-08-001: Credential Dumping via Windows Task Manager ↗MITRE · MITRE · ExternalKnowledgeBase
- CAR-2013-10-002: DLL Injection via Load Library ↗MITRE · MITRE · ExternalKnowledgeBase
- Deterministic method for detecting and blocking of exploits on interpreted code ↗K2 Cyber Security Inc · Jayant Shukla · PatentReference
- Hardware-assisted system and method for detecting and analyzing system calls made to an operting system kernel ↗Endgame Inc · Matthew D. Spisak · PatentReference
- Malware detection in event loops ↗Crowdstrike Inc · Daniel W. Brown · PatentReference
- Post sandbox methods and systems for detecting and blocking zero-day exploits via api call validation ↗K2 Cyber Security Inc · Jayant Shukla · PatentReference
Bare Metal Cyber is an independent educational publisher and is not affiliated with or endorsed by The MITRE Corporation. MITRE D3FEND™ and the D3FEND logo are trademarks of The MITRE Corporation. MITRE ATT&CK® and ATT&CK® are registered trademarks of The MITRE Corporation. Use of D3FEND source material is subject to the official Terms of Use.