A common perception associated with AI/ML in cybersecurity is that it's a market of lemons, and Gen AI will just be another buzzword that every single vendor says they have but doesn't mean anything to end-users. However, I actually believe Gen AI offers a rare opportunity to reset this perception because most of us can quickly tell if a Gen AI-powered product capability (e.g., chatbot or copilot, assistant, or analyst) is smart or dumb and helpful or a waste of time within a few minutes of interactions. This is not the case with AI/ML-based detections. You wouldn't believe how many times the detection product I previously worked on was called snake oil when a BAS product was brought in to test us, and we didn't detect anything 😂 because: 1. data exfiltrations that involve transferring 1kb files to another internal IP 2. brute force attacks that involve a single attempt 3. network recon with a single unestablished TCP connection 4. share enumeration with a single failed SMB connection 5. corrupted attack pcaps due to bugs in the attack replay tool 6. C2 beaconing that lasted less than 1min to an internal IP AI/ML amplifies the hard-to-test nature of most security products to the next level because it's so difficult to actually craft realistic environments required to differentiate most ML from simple rules. For example, MITRE EDR tests have very little baselining and less than 6 VMs/hosts, which incentivizes building "ML-based" hair triggers, such as flagging anything non-zero. Over time, AI/ML-powered detections slowly degraded to a market of lemons perception because nobody could actually tell whose ML was real. I look forward to Gen AI maturing from buzzwords to practical product capabilities that provide tangible and easily perceivable benefits in cybersecurity. This will transition the community's perception from skepticism to optimism and perhaps finally wash away the "market of lemons" stigma surrounding AI/ML for cybersecurity in the last decade.

If you have ever wondered which security vendor has real AI technology and which vendor is selling snake oil 🐍, here are two "inside baseball" questions you can ask the vendor to determine who is swimming naked 🏊‍♂️. AI is ubiquitous in cybersecurity vendor marketing. However, the term has become increasingly associated with vendor snake oil because most buyers cannot see through the marketing bluff. As someone who built an AI/ML detection product from scratch, I want to help you become a more informed buyer 🤓. Disclaimer: All the examples below are entirely fictional. Any resemblance to actual vendors or products is purely coincidental. 1️⃣ Where exactly are you using AI in your product, and can you give me a high-value real-world example? The most common form of overmarketing is claiming that a product is AI-based when only a tiny (or sometimes immaterial) portion uses AI. For example, an AI-based XDR product might use a single ML model to detect a specific attack while employing 1000 different regex for most other detections. Another more recent example could be a vulnerability management vendor claiming to be powered by Gen AI when all they do is take a CVE number and a list of IPs and use ChatGPT to write a sentence combining them. By asking for specific examples, you are probing whether the vendor is using AI in areas that really matter and in volumes that can lead to significant efficacy improvements. One thing to remember is it's totally fine to only use AI in some places because many product capabilities should not be built with AI. The goal is to have vendors really clarify where they are using AI and determine whether additional efficacy in that area is important to you. 2️⃣ How does your AI automation adapt to complex and unexpected scenarios, and can you give me a real-world example? Another typical sleight of hand in vendor marketing is calling playbooks/scripts AI. Playbooks and scripts (like dishwashers and washing machines) can deliver reliable and meaningful automation by following pre-programmed steps and conditions. However, they are not AI because they can not adapt to complex or unexpected scenarios (such as someone putting a watermelon into the dishwasher and expecting it to adjust its program and spray patterns accordingly). One example would be an AI alert triaging product using a set of playbooks to extract IOCs and look them up in TI feeds. This surface-level automation of IOC enrichment might be sufficient in certain environments. But, due to the variability and complexity of security alerts, playbook-based triages are significantly less thorough and complete than a human or truly AI tier-1 analyst, which might be very important to other buyers. By poking at how the product behaves in complex and unexpected scenarios, you can quickly work out if the "AI automation" a vendor is offering is based on playbooks and scripts (wearing AI hats) as they are completely rigid and reliant on the pro-programming.

If you don't have a perfect SIEM deployment, and your security data is scattered across a dozen security tools and data sources, we have a new solution for you: dropzone.ai/blog/how-the-dro… No one has a perfect SIEM deployment where all the data the SOC ever needs is perfectly normalized and curated in the SIEM. However, pulling data from individual security tools and data sources takes a lot of work. Tools are complex and often riddled with quirks and domain-specific query languages that require non-trivial expertise. I bet you don't remember how to write an LDAP search filter that looks for the manager of a specific employee without Googling it. Instead of pursuing the security data engineering nirvana, we are introducing a new alternative approach: building a system that integrates with your entire security tool stack and knows how to query every tool, similar to how human analysts manually stitch data across multiple tools today. SOCs currently require their analysts to know so many details, from SPL syntax and schema of common indices to Wireshark display filter language, and analysts spend more time dealing with tool nuances than real, meaningful work. It's time to uplevel the abstraction that analysts operate at and reduce the overhead of data segmentation and tool usage.