With all of the big data sets and volumes bogging down forensic examinations nowadays, it’s nearly impossible to keep up with caseloads unless you have an excellent triaging methodology or a Processing Engine. Inability to handle large data sets can create a backlog of unprocessed examinations. Frustration is soon to follow.
In order to combat potential backlogs, it’s better to utilize systems geared towards processing power. Forensic Workstations primarily rely on forensic write-protected devices and offer potentially slower processing speeds. For example, multiple 6, 8, 10, 12TB drives can slow down the ingestion and culling of data.
Processing Engine systems are specifically geared towards high-speed calculations. Sometimes there is enough headroom to allow other processes running in tandem. The key is hinged on fast SSDs (PCIe NVME’s – PCIe Non-Volatile Memory). However, there are better solutions that can be implemented. This includes RAIDing PCIe NVMe devices through various hardware. Furthermore, the RAIDing of SSDs and 12Gb/s SAS drives also produces speeds that qualify as a processing engine.
The Processing Engine is described as such:
“The processing engine is responsible for processing data, usually retrieved from storage devices, based on pre-defined logic, in order to produce a result. Any data processing that is requested by the Big Data solution is fulfilled by the processing engine. 1”
Practitioners are utilizing Processing Engines to perform individual tasks. They are piecing together their examination on the results provided by individual systems. Think of “divide and conquer” where numerous components are divided among workers. These individual tasks are combined so the final product is a culmination of the workers involved.
Speed and Performance
So what type of speeds should you expect and/or shoot for with your Processing Engine? By RAIDing together PCIe NVMe’s, you should be able to achieve speeds over 12 Gigabytes per Second. SSDs drives typically run at either their specified limitation or perhaps faster depending on the RAID Controller. These processing drives, if you will, feed software the speeds required to run as fast as possible. It’s critical to know how to configure the Processing Engine for each client. Some software packages on the market can run conventionally faster with RAIDed SSDs. This software recognizes all of the drives connected to an SSD RAID as individual disks. These RAIDs utilize those disks to create indexes for the case.
If you are considering a new “workstation” for your laboratory, perhaps you should consider implementing a Processing Engine rather than a “forensic workstation”. It is critical that the designer of your system understands the implementation of a Processing Engine. The benefits are considerably faster and will allow you to get through Big Data sets more efficiently.
1 Arcitura; https://patterns.arcitura.com/big-data-patterns/mechanisms/processing_engine, retrieved 6-January-2020.