A hands on introduction to MattockFS
Rob J MeijerDFRWS-EU; March 21 2017; Überlingen, Germany
While the growing use of triage in the computer forensic process has mitigated
the growth of the amount of data reaching computer forensic labs, and while SSD
technology result in largely CPU restrained forensic data processing for small
size investigations, for medium to large investigations the use of traditional
hard-disks remains dominant and combined with advanced in CPU processing
power, has shifted bottlenecks from being largely CPU based to being
increasingly more IO based. The pervasive use of secure hashing in the lab-side
forensic process combines CPU bound aspects important in small scale
investigations run with SSD technology as well as IO bound aspects in medium to
large investigation run with traditional hard-disks. Further, anti-forensics form a
growing concern in (semi-)automated forensic processing. MattockFS aims to
provide a local message-bus and data-archive building block for use in (semi-)
automated lab-side digital forensic media-data processing. A building block that
considers IO concerns that come with message-bus based asynchronous
processing, hashing related performance concerns and anti-forensics related
integrity concerns. The presented building block will be illustrated with both a
native and a python based walk-through, which the attendees will be able to
follow hands-on using MattockFS on their laptops.
The intended audience is digital forensic practitioners and researchers. Some
investigative experience, and a working knowledge of Linux and Python is
required. A familiarity with semi-automated lab-side processing as well as
asynchronous data processing models would be an advantage although not
If you are going to attend the MattockFS workshop at the DFRWS-EU in Überlingen,
it is important to realize that while the part before the coffee break will consists
of presentations that require no preparation on your part, most of the post coffee break
part of the workshop will be very much hands on. Please bring a laptop and either prepare
your laptop or a virtual machine on your laptop to run Ubuntu-16.04. If you have no personal
preferences regarding virtualization software, VMware is suggested. Further, we will be
using a simple EWF disk image that you may want to download before the workshop as not to rely
too strongly on available bandwidth during the workshop. You have a number of options regarding
preparations for the workshop:
- If you are already running Ubuntu 16.04 on your laptop as host system, you may suffice with downloading macwd.E01 to your laptop.
- You may install Ubuntu Server 16.04 on a VM on your system. I would like to suggest allocating at least 100 GB to your VM so you have some room to work with. After installing Ubuntu on your VM, please run the following commands:
- sudo apt-get update
- sudo apt-get upgrade -y
- wget http://dfrws.capibara.com/macwd.E01
- If you are reading this last minute and have come unprepared to the workshop, you may download a VirtualBox OVA file. This file is 1.5 GB in size. The instructor will have a small set of flash drives available, but if you can manage to download it before the start of the session that should be prepared as there may be other participants that need to use these USB keys as well. When using an USB passed along by an other participant, please check the hash of the files are unchanged! Note that the OVA file comes with macwd.E01 included and already has all the packages installed described in the optional preparation section below. You can log into the VM as user 'dfrws' with password 'dfrws'.
Optional participant preparation
After installing Ubuntu, there is a small list of standard packages you need to install that are either dependencies of MattockFS or include tools that we will be using during the workshop. You don't need to do this before the workshop, but it doesn't hurt either and might save some bandwidth from fellow attendees who didn't have time to complete the following steps yet. The following command should install the dependencies:
Now you should have everything you need apart from MattockFS itself. We fetch the latest git snapshot from github.
- sudo apt-get install -y fuse redis-server python-fuse python-redis python-xattr python-libewf sleuthkit git python-setuptools libpython-dev exif binutils python-demjson python-exif attr python-magic
- wget http://dfrws.capibara.com/python-fadvise_6.0.0_amd64.deb
- wget http://dfrws.capibara.com/python-pyblake2_0.9.3_amd64.deb
- sudo dpkg -i python-fadvise_6.0.0_amd64.deb
- sudo dpkg -i python-pyblake2_0.9.3_amd64.deb
Don't do anything else yet, we shall be doing the rest during the workshop, and it is important we all have the same snapshot then.
- git clone https://github.com/pibara/MattockFS.git
The workshop is logically divided into five parts. The first part serves
as an introductory lecture covering asynchronous message passing concurrency
and spurious read issues (including page-cache misses) incepted by combining a
message passing concurrency model with a tool-chain model of processing. This
first part also covers integrity concerns in non-monolithic systems that could
make a whole framework vulnerable to anti-forensic vulnerabilities in a single
component. We discuss the use of the capability model of security and other
privilege separation techniques important to high integrity system design.
In the second part, we shortly look at the most direct ancestors of MattockFS:
CarvFS and MinorFS and discuss the use of CarvPath annotations as central
In part three we then see how the concepts from part one and two come together
in the MattockFS design.
Part four is the last theoretical part of the workshop where we outline how the MattockFS building
block might be used in the construction of a distributed forensic processing
The fifth and final part of the workshop, that should take up most of the post-coffee-break session,
is the hands on part where we walk through the installation and (programmatic) use of MattockFS.
- Asynchronous processing and the tool-chain approach.
- Integrity, privilege separation and capability based security model
- CarvFS and MinorFS
- MattockFS core design.
- MattockFS as distributed-framework building block.
- Installation walk-through
- File-system as API walk-through
- Python API walk-through: a naive module
- Wrap-up discussion
Slides for the workshop
Below are PDFs of the slides for the workshop:
Some reading material
MattockFS started off as proof of concept refference implementation created within the context of a M.Sc research project. The minor thesis for this research project is available on Research Gate. Apart from this paper, you might be interested in some old material regarding systems that MattockFS was based upon: