https://retiari.us/categories/response/ Recent content in Response on Alex Maestretti Hugo -- gohugo.io en-us Thu, 07 Mar 2019 19:27:40 +0000 https://retiari.us/posts/ir_in_la/ Thu, 07 Mar 2019 19:27:40 +0000 https://retiari.us/posts/ir_in_la/ I am excited to announce the latest opening on our Security Incident Response Team (SIRT), which is also our first LA-based role. Netflix is working hard to provide a wide range of content choices for our customers and with all that business activity comes some amount of work for the SIRT. In this role you will be exposed to a high operational tempo with a lot of growth and ambiguity. It is a very independent role where you will set the agenda for how SIRT can best add value in content related spaces. That is not to say you will be without support! You will be a core member of SIRT with peers in Los Gatos to help with training, tooling, and crisis management, as well as a growing Studio Information Security team next to you in LA. This role will be part of the SIRT on-call rotation so you will respond to issues across all lines of business while on call. We work hard to keep our on-call burden reasonable and SIRT members are expected to make progress against longer term objectives to ensure on-call stays engaging but not overwhelming. SIRT members tackle a range of projects from programmatic improvements to our response, training our team and others on how to respond better, developing tooling to automate crisis management and forensic activities, and other cross team contributions. https://retiari.us/talks/first_2018_podcast/ Thu, 07 Jun 2018 12:56:35 -0700 https://retiari.us/talks/first_2018_podcast/ A podcast for FIRST 2018 on how the Netflix SIRT operates. https://retiari.us/posts/memory_forensics_cloud_containers/ Wed, 11 Oct 2017 12:59:25 +0000 https://retiari.us/posts/memory_forensics_cloud_containers/ At Netflix my targets are primarily Linux VMs running in a microservices environment. I rarely have to pursue criminal prosecution but often want to figure out what is happening. Much of the work in memory forensics has targeted Windows systems and forensically sound capture methodologies that pull the entire system memory. This focus makes sense given the attack surface that user systems represent; however, given my goals I believe that userland memory acquisition and targeted analysis of memory at process granularity could yield more efficient results. Microservices, CI/CD deployments of immutable and ephemeral instances, the trend to containers, and various hardening efforts in the kernel, confer advantages on the defender that we must leverage to keep up with the scale and velocity of our environment. Another area that is not well developed is when target processes on the edge manage their own memory, as many of today’s popular languages do. I am curious what folks are doing in this space, so I wanted to mention some trends I see and solicit feedback: Desktops vs Datacenters vs Microservices User systems are the most difficult to secure. Particularly general compute devices operated by a technical workforce that needs full access to their systems, and will install all manner of tooling from around the web to accomplish their jobs. https://retiari.us/posts/uservice_for_disk_forensics/ Wed, 04 Oct 2017 12:55:22 +0000 https://retiari.us/posts/uservice_for_disk_forensics/ This is a follow-on to my previous post here. In the last few months I have had the chance to talk to a number of other companies and gotten a lot of great inbound response to the post about disk forensics in the modern era. To summarize: “Our goal is to move towards a streaming model for disk forensics by updating various existing tools that are built around the concept of imaging a single machine and creating a file. In a cloud environment we need to image multiple (even hundreds) of machines and process all those images into meaningful event data. Acquisition is fairly painless via API, but then what to do with the resulting snapshots?” While we were hoping for a cloud agnostic solution it seems we will need to solve this locally first and then seek areas to reuse code and integrate our systems in the future. We continue to approach this problem by decoupling acquisition, processing and analysis. We also hope to implement this in a scalable manner, ideally leaning into serverless techniques to minimize infrastructure maintenance. Acquisition Google is investing in DFTimewolf as part of the log2timeline/Plaso, along with GRR and Rekall. I think the Rekall agent architecture is the way to go. https://retiari.us/talks/aws_forensics_reinvent/ Mon, 07 Aug 2017 12:56:35 -0700 https://retiari.us/talks/aws_forensics_reinvent/ A talk from Re:Invent 2017 on how to look for evidence on disk in AWS. https://retiari.us/posts/disk-forensics-pipeline/ Mon, 22 May 2017 12:05:48 +0000 https://retiari.us/posts/disk-forensics-pipeline/ Last week we hosted colleagues from Google, AirBnB and Adobe to work on a joint effort to improve our disk forensics capability. Our goal is to move towards a streaming model for disk forensics by updating various existing tools that are built around the concept of imaging a single machine and creating a file. In a cloud environment we need to image multiple (even hundreds) of machines and process all those images into meaningful event data. Acquisition is fairly painless via API, but then what to do with the resulting snapshots? We are working towards a cloud agnostic streaming service for disk forensics. To get there we are currently focusing on acquisition in AWS and then sending those snapshots through Plaso for processing, pushing results into ElasticSearch, which is then fronted by Timesketch for analysis. I filed a number of issues with Timesketch to move us towards this, and we are working with the Plaso maintainers as they do some refactoring as well, but even now, with a few hacks, we can compose a system that achieves our goals. Acquisition Currently we have a set of boto scripts that enables us (Netflix SIRT) to capture snapshots of a group of instances within AWS. https://retiari.us/posts/fixing-ir/ Fri, 16 Sep 2016 12:11:02 +0000 https://retiari.us/posts/fixing-ir/ What makes detecting and responding to malicious activity on information systems so difficult? Incident Response and investigation can be incredibly intellectually rewarding. You get to puzzle together what transpired and take action to defend your systems, while working against a wily and adaptive adversary. On a bad day, however, you will find that most of the puzzle pieces are missing and there is no wily adversary present, only a deluge of false positive security events being happily emitted by an expensive PRNG called a SIEM. The core issues are volume of activity on, coupled with the complexity and change of, the systems you are defending. Traditional IT management compounds these issues rendering effective response impossible. DevOps, microservices and cloud infrastructure, in solving for scalability and development velocity, also offer solutions to these core security problems, and have the potential to Make IR Great Again. IR allows you to exercise a wide range of security skills. The diversity of events that one encounters ensures a varied and engaging workflow that continues to challenge. It takes not only a keen intellect and technical skill, but a body of experience from which to draw. Often there is no runbook, so you cobble together inferences from similar past cases and drive on. https://retiari.us/posts/aws_ops_rig/ Mon, 01 Aug 2016 12:46:56 +0000 https://retiari.us/posts/aws_ops_rig/ Every responder needs a solid platform from which to operate. Something fast, secure and pre-loaded with relevant tools. Conducting incident response in a public cloud is the same as traditional IR but different. The public cloud is after all just other people’s servers; however, the ‘other people’s’ part changes things up a bit. You can’t roll into Amazon’s data center and grab a disk out of the rack, but Amazon provides an API for that. The concept they have put forward is ‘shared responsibility’ in which they handle the hypervisor and below, and above that is on you. We are moving towards a serverless approach to forensic response leveraging lambda functions and Amazon APIs; however, there is always a need for a solid ops box stocked with tools and enough permissions to get the job done. Design Goals: AWS EC2 native - leverage AWS APIs; minimize external dependencies - can’t rely on much in a crisis. Ephemeral instances - Spin up fresh, clean and quick; and kill them off once needed data is persisted to S3. Tools for response on Debian/Ubuntu AMIs with headless server (no GUI). Use corporate services day-to-day, but provide fallback options if they are down.