SIEM’s value proposition has evolved from compliance reporting to forensics and now detection.
Centralized data facilitates global visibility across control points, enabling effective detection rules and threat intelligence integration.
Transcript:Travis LanhamWhat’s your take? Yeah, thanks for having me. I think it’s a great question, and I think it kind of starts with what’s the value that folks get out of a sim today, right? Originally, this value started compliance reporting, right? Put a bunch of your logs in here, and we’ll get you that checklist of whether you’re meeting certain compliance regulations, right? Then I think this kind of evolved into security practitioners jumping into the tools and getting value out of sims for forensics, right? Okay, I know I had a security incident. Maybe my antivirus told me something, you know, look bad, and I want to go and do a more thorough investigation. And then I think on that maturity curve, there was kind of the next advancement around actually using SIMs for detection. And I think this is really where SIMs are still today, right, and trying to prove out value around detection. And I think if you look at what you need in order to provide value from detection in a sim that’s not in a control point, right? And being able to kind of aggregate this global view across all your different control surfaces, whether they be endpoint, network identity, etc, is around how can we bring data together? How can we have that, you know, in a common format, have that in something that detection rules can run over and have that be something that can be queried and metrics can be driven out Of to be joined against threat intel and kind of all these use cases. And so I think that really comes back to if that’s kind of the next value prop for SIEM, right? And some organizations achieve that to an extent today and a lot don’t, right? Then I think it comes to this question of, okay, where does that data need to live? And how does it need to be operationalized to achieve that goal? And I think that brings this into kind of an interesting set of questions where today, I think a lot of those use cases require all that data to be in one place. That data today lives in many different places, which is why I think this topic has become very popular. And I think there’s this open question on, hey, in the future, is this kind of going to be a lot of data still needs to be centralized? Can some of it run more at the edge of where your data lives? I think those are kind of the key elements.Tim PeacockSo help me understand, you’re proposing a world where you can achieve centralized visibility without centralized data? I think that’s the goal of this kind of disaggregation or shimming on top of other data lakes approach.Travis LanhamI don’t think that it’s very effective today. I think a lot of the why folks are trying out this approach is because the number of control points that they have generating logs or generating telemetry or visibility into their environments Has grown a lot. And their security teams or a operational management teams haven’t been able to scale out their kind of centralized visibility as quickly, right? And so often kind of falling back to that second value add, right, around forensics, right? Often, you know, what we’ll see folks trying to do is, hey, I need to go and do forensics in this cloud environment and the logs for that cloud environment live there. And that might turn into, okay, I need to run some queries against that data, or I need to pull that data into my SIM and then run some queries there. But I think it’s still kind of stuck in that forensics security, right? It’s still not getting into that true vision for SIM as being this kind of central detection point where you can really get a lot of metrics that will help you do kind of more threat intel Driven approach to detection that will help you ultimately become what I think that first compliance milestone was around. You have the forensics, but then with the detection, you could also potentially get into this, you know, a future state of the sim as being this kind of governance control point, right, Where you can not only have compliance as a reporting capability, but you could have kind of this thing that sim is naturally positioned well for because it has all this visibility, Which is to actually have, hey, here are the compliance controls that I want to enforce across my environment. I have all the monitoring visibility that telling me, am I meeting those or not? And then I can hook into kind of the remediation side or the prevention side and actuate out in my environment. Here are the controls that need to be enabled, you know, two-factor authentication, you know, binary denial listing, whatever it might be, right? And kind of have that, you know, be that value add of having that central view of the data. And so I think it’s like today, a lot of that requires having all that data in one place. And I don’t think that, you know, there’s really been an effective way to get that central point of visibility without being able to put your data all together. (Time 0:02:41)
Centralization Challenges
Centralized data platforms struggle to keep pace with expanding attack surfaces.
Forensic investigations in cloud environments often necessitate pulling data into SIEMs, hindering real-time detection.
Transcript:Anton ChuvakinWait a second. That to me doesn’t automatically mean we have to decouple. I mean, in theory, you can build a platform that combines high-scale storage. Oh, no, I guess I’ll cut off myself in the middle of the question because if you’re doing it federated, you probably can’t really truly couple it, right? So that’s the argument?Travis LanhamI think there’s one kind of piece that’s around things like performance and like, can you really do a lot of these powerful metrics across all your different visibility points, etc. Right? I think there’s another piece of this, which is really around whether if you have different data sources in your sim today, and you know, a lot of people have kind of tried different approaches To normalization, like, is that normalized format able to give you something that I can ask a question, and it goes across all my log sources and gives me the answer, right? Even a simple question like, have I seen IP address x in my environment, right, that requires doing that kind of basic level of normalization, that if you have a sim today, and you have Multiple data sources in your sim today, a lot of folks aren’t even getting to that kind of maturity level of, okay, I have even these most basic parts of those security logs parsed into A consistent manner, right? So I think that’s almost, if we’re not doing really well at that problem today, in the industry, then trying to do that across a federated approach, where all the data isn’t even in the Same system, I think becomes this exponentially more difficult problem. (Time 0:07:12)
Decoupled SIEM Considerations
Consider the upsides of decoupled SIEMs, such as leveraging existing storage solutions and focusing on security.
Acknowledge potential drawbacks like assembly complexity and performance issues.
Transcript:Anton ChuvakinSo wait a second. So people who insist on decoupling or sort of disconnecting the security part from the data storage part, like, what are the upsides? Like, do you have compassion to these people? Or do you think they’re wrong? Or like, how do you think about not trying to lead you to an answer? I’m just like, just curious, because I see those industry debates when people say, no, no, no, no, no. Now we have this amazing storage. We can just build security, plug it in into the scalable storage, whether it’s BQ or something else, and we’ll have a SIM better than anything that came before, even though it’s technically Decoupled. And these are tempting, but I also have this flashbacks to the time of the old times when, sure, you can buy an Oracle database off the shelf and then build on top of that. And then you would have a decoupled sim, but that approach we abandoned. So I’m schizophrenic about this, torn, really torn about this. (Time 0:08:33)
Google’s SecOps Approach
Google’s SecOps prioritizes centralized visibility for proactive threat detection and response after the Aurora attack.
This approach facilitates innovation and threat intelligence integration for a comprehensive security posture.
Transcript:Tim PeacockBut like thinking about it, and you’ve been at Chronicle a while, or sorry, SecOps. Well, you were at Chronicle for a long time and you’ve been at SecOps for a little while. Tell us about why we haven’t gone this direction as Google Cloud. Like why do we persist in this central and consistent so that you can do easy, like, I want to find this IP, that’s a single line query? Why haven’t we gone this other direction?Travis LanhamYeah, I think for us, kind of the motivation starts around where Google has come from solving this problem, right? And where we’ve come from solving this problem, you know, even in response to Google’s own Aurora attack a decade ago, right, was we really need to get centralized visibility into all These different surfaces that we have. And once we have that centralized visibility, now we’ll be able to not only achieve kind of that basic compliance capability, not only that forensics capability, but actually be able To go and innovate in that detection capability or that higher order capability. And so where we’ve come in delivering the product out to customers is trying to get them to that same point, right? And this comes out in a lot of different ways in the product, where we do things like thread until matching and retroactive IOC matching, where we have your detection content that goes And runs over log for your data. In order to do that, we really need this data to be in one place, we need it to be in a consistent format. And so that’s been a big area of focus for us initially. I think kind of looking to the future, right, I think there are kind of all these areas of expansion that we can go into, right? And we’ve already done some of this with SOAR, for example, today, where in SOAR, you can kind of do this federated forensics use case and tie that back into SIEM very closely and get the Threat Intel view on that data. And so I think there’s kind of been like the first steps toward it. But I think it still comes back to this. The hard part is around having consistency in your data, right? Having a consistent view of the data where you can now do analytics, right? And then with that baseline of analytics, you get detections, you get threat intel kind of being proactive view. And then ultimately, I think you’ll be able to get into this compliance plus plus of the governance, right? And be able to get into this, okay, we have this great view of visibility. And now we can actually start actuating the policy that we want to enforce in our environment. (Time 0:11:56)
Coupled SIEM Advantages
Tightly coupled SIEMs offer integrated data consistency and best-in-class capabilities for security data operations.
Features like real-time search, detection, and aliasing enhance threat investigation and noise reduction.
Transcript:Anton ChuvakinCan also be decoupled and still do exactly that right in theory you can have the off-the commercial data store and then assume that it’s there at the client site and then just build the Security layer so presumably when we decided to not do that we saw other upsides of a tightly coupled data store plus sim plus security experience so maybe you can talk about these like What are the what are reasons but consistency? Because you can achieve consistency with VQ or whatever, as a data storage backend, we didn’t do it, though.Travis LanhamDefinitely. So I think there’s kind of that baseline, right, of like, why kind of start with this approach, right? And it gets a lot of those pieces, like, okay, we have central consistency over the content, we have central consistency over, you know, the parsing and detection rules and the metrics That are generating kind of all those pieces that are core building blocks of the platform. And so I think that’s, hey, the easy way to get that right, or the way to get that in a more comprehensive fashion is where you can have end to end testing over all that content, you can have This kind of very cohesive story where it’s all integrated well together, right? So that’s kind of the starting point, right? And then on top of that foundation, right, we’ve really built out a series of best-in capabilities for operating over your security data that just wouldn’t be available in an off-the Database or, you know, a Parquet file set that, you know, folks are putting their Databricks instance on top of or, you know, putting in some data into Snowflake, right? And so a lot of those capabilities we’ve driven around are how can we bring Google search to your security data, right? How can we kind of get out of this method of having tons of indices or these folders of data and provide this kind of, hey, I just want to look at this alert. I want to put in an entity in my search. I want to see everything that we have around that, the threat intel overlay on top of that. I just want this kind of unified investigation experience and make that as easy as going and doing a Google search, right? And in order to do that, building on kind of Google’s great foundational infrastructure, we’ve built a bunch of application platform infrastructure on top of that in order to support Things like real-time search, (Time 0:14:07)
Noise Reduction with SecOps
Customers using SecOps achieved significant noise reduction in alerts, like unauthorized service account key downloads.
This was possible by simplifying complex lookups and unifying data, freeing security engineers for proactive threat analysis.
Transcript:Travis LanhamAnd so we’ve had customers, for example, that have been able to achieve dramatic noise reduction in their alerts because I’ll use an example where it’s, hey, I wanted to alert on any Time someone downloads a service account key for my cloud environment, right, that’s not part of the SRE or IT team. And before for them, that was kind of this complicated mix of what Anton was talking about, right? Maybe I have some lookup tables. Maybe I have some things that happen in SOAR, right, as kind of these enrichment playbooks. And it’s hard to keep all of this consistent. It’s hard to have this kind of, now you’re responsible as a security team, not only for doing this, okay, I need to load that data into this place. I need to keep that data consistent for, here are all the people in my organization, here’s their job role, et cetera, right? But now I also need to keep that consistent with my detection logic. And so now you’ve turned the security engineer’s job into, hey, your job isn’t to investigate threats and look out proactively for the risk of the organization. But your job is to be this human extracts, transform load pipeline of your CMDB data into your security system and then try and maintain consistency with all these playbooks. And kind of it just becomes this complicated mess rather than doing in kind of this. Hey, you just bring your data in. All of this gets enriched together, and we present this unified picture, right? And I think that kind of, again, connecting it to the beginning, right, is a lot of these kind of capabilities, Google search over security data, always on enrichment, you know, detection At really unlimited scale, these really wouldn’t be possible unless you have a platform that’s built for security, rather than just taking some Parquet files that are in some cloud Object storage. (Time 0:20:02)
Decoupled SIEM Limitations
Decoupled SIEMs often focus on interface and content, lacking innovative processing capabilities.
This limits advanced functionalities like efficient search, noise reduction, and prioritization for improved security maturity.Transcript:Anton ChuvakinCorrect. I think that it has to do with one of the lessons from Google, sort of like how innovative products ultimately require or the best innovative products require technical innovation. So they need to be an invention under the hood. It can’t be the, oh, we’ll just speed it up a little bit or we’ll just do this. So in your mind, if you’re thinking about sims that are trying to decouple from data stores, what may be the technical innovation driving them? Like, I’m a little puzzled here.Travis LanhamI think that that’s kind of one of the fundamental issues, right? That if you’re trying to be this shim layer, right? Ultimately, a lot of the value that you’re providing is kind of interface at the interface level, right? Or it’s ultimately a kind of the content level, right? And trying to stitch together this kind of different set of data silos, rather than providing a really innovative processing capability on top of it, right? And so it’s fine as kind of this, hey, I want to be able to see all these different parts of my environment, right? Or I want to be able to extend into these places where I don’t have stuff set up, right? And I wasn’t bringing my logs into, you know, the place that I want to, right? But it’s not really getting to that. How do I have amazing search over my data? How do I have all these capabilities that allow me to, you know, have better noise reduction, have better prioritization, kind of all these more capabilities up the maturity curve. And I think this is also why folks who have kind of gone down this road, right? Of, okay, yeah, I’ll just keep my logs in cloud object storage. And I don’t need to really worry about bringing them into a SIEM. It’s like ultimately when they find themselves in an incident, they find themselves, okay, now I need to pull these (Time 0:21:51)
Data Management in SIEM
Centralized SIEM simplifies data management for security teams, avoiding data infrastructure tasks during incidents.
Decentralized approaches can burden security teams with data handling during critical times, hindering effective response.
Transcript:Anton ChuvakinSo that’s not fun at all. What about the argument that allows the security people to deal with security and leave the data storage to quote-unquote data people? Like the argument that if I have a team of 20, I can dedicate 20 to security and have the storage to be magically done by somebody else who I pay or who my clients pay.Travis LanhamWhy does this argument not hold water? I think this is the argument for not going in the kind of decentralized version of this, right? Or not going in the disaggregated version, right? Because if you have that kind of centralized point and you have someone who’s responsible for the storage, right? Then your security team doesn’t need to be worrying about that, right? If you have this kind of, okay, I have all these different data silos, right? And then you have the incident come through. Now your security team is often playing that data infrastructure role, right? Or again, kind of at the worst possible time, you’re throwing in a whole nother part of the IT organization to try and help the security team, right? And now you have not only the security incident itself and the coordination that the security team needs to be doing with all the folks, you know, they’re either impacted by the incident Or, you know, that are the trigger of the incident or whatever, right? Or the exposure of the incident, right? But now they have to be coordinating with this team that’s actually just trying to like get all the visibility into one place. I have been trying since Anton asked that to think of a suitable (Time 0:23:42)
SIEM Maturity Assessment
Assess your organization’s SIEM maturity level to determine the best approach.
Focus on evolving from basic compliance to advanced risk management and proactive threat hunting.
Transcript:Tim PeacockMetaphor and I have failed, which maybe means sadly, it’s time for the end of the episode. I have to ask our traditional closing questions, Travis. One, do you have a tip to help people think about the future of Sim and which way they should go as an org? And two, do you have recommended reading? Because we hate to leave listeners empty handed.Travis LanhamYeah, so I think the first one is really kind of thinking about that maturity curve, right? And where are you on it, right? Are you doing kind of checkbox compliance? Are you doing kind of more around the forensics? Or are you starting to build out detection capabilities? Are you starting to build out you know, and I think this is even more for lean kind of some of those governance controls, and really taking a risk lens on top of your security data, right? Because ultimately, this is where organizations are orienting towards now, right? Where it’s, hey, I’ve gotten all this IT infrastructure, security is top of mind. And I really need to be thinking about how do I prioritize and manage the risk of my attack surface, right? And so I think that’s, you know, where we see a lot of folks putting a lot of energy around today. And then for required reading, I think there are a bunch of different sources. Suggested reading. Yeah, yeah. Instead of frequently, recommended, frequent demands, kind of that logic. Yeah, recommended reading. I mean, I think there are lots of great blogs out there. The things that I find actually most interesting usually are just when folks post an instant report, right? You’ll have Cloudflare, you’ll have Snowflake or whoever, right? Posting an instant report or getting a threat report, whether it be from Google Threat Intel or some other source, right? I think those are usually kind of always interesting, right? Because you really kind of start to get into the heart, especially for the ones that are quite well written, or Oak does another example, right? Where organizations who have done a service to the broader community by really writing up, hey, here was our experience. Here’s things that you should be looking for, right?Tim PeacockAnd here’s kind of a view on how this happened to us and how you can think about preventing it for yourselves. I think that’s such a great way of looking at an after action report of here’s how you can avoid having this happen to you. But (Time 0:25:00)
Security Data Lakes
Security data lakes are envisioned for next-gen SIEM scalability and custom analytics, especially in mature organizations.
Centralizing data in a data lake architecture enables scalable performance and advanced analytics capabilities.
Transcript:Anton ChuvakinBefore we go, before we finish, we managed to run this whole episode without once mentioning the security data lake. I just am curious how they connect to this whole thing, because you see people enamored with the concept, but you don’t always see people who understand what they’re talking about, Right?Tim PeacockI see. Okay, Travis, what’s the security data like? And why are people enamored?Travis LanhamYeah, I think at least when I’ve heard folks discuss it, it’s mainly been a couple of contexts, right? One context is around this idea of next generation SIEM, right? Or SIEM built for scale, right? And being able to put all of your data into a SIEM and have this data lake architecture that can really scale up beyond some of the approaches that have been tried earlier in the industry That you kind of mentioned, whether it’s taking off the shelf single node database or taking a single rack server, right? To really get into this data lake of, okay, it’s horizontally scalable and you can kind of decouple the compute and storage, right, and really get your great performance, etc. That way. I think the other way or place that it comes up is basically in this folks wanting to do their own security analytics, right, or folks wanting to have their own, you know, Hadoop infrastructure And do security analytics themselves, I think, particularly for some of the really mature tech or kind of government organizations, right? And I think in there, you know, that kind of is, you know, I think it’s very related in order to do that. And if you want to have your own analytics jobs and things like that, you probably want to have your data in one place. And so I think that kind of ties into a lot of the things that (Time 0:27:00)