Malicious open source packages are no longer edge cases. They are deliberate, scalable attack paths targeting developers, CI pipelines, and the software supply chain itself.
⬥EPISODE NOTES⬥
Modern application development depends on open source packages moving at extraordinary speed. Paul McCarty, Offensive Security Specialist focused on software supply chain threats, explains why that speed has quietly reshaped risk across development pipelines, developer laptops, and CI environments.
JavaScript dominates modern software delivery, and the npm registry has become the largest package ecosystem in the world. Millions of packages, thousands of daily updates, and deeply nested dependency chainsഴ് often exceeding a thousand indirect dependencies per application. That scale creates opportunity, not only for innovation, but for adversaries who understand how developers actually build software.
This conversation focuses on a shift that security leaders can no longer ignore. Malicious packages are not exploiting accidental coding errors. They are intentionally engineered to steal credentials, exfiltrate secrets, and compromise environments long before traditional security tools see anything wrong. Attacks increasingly begin on developer machines through social engineering and poisoned repositories, then propagate into CI pipelines where access density and sensitive credentials converge.
Paul outlines why many existing security approaches fall short. Vulnerability databases were built for mistakes, not hostile code. AppSec teams are overloaded burning down backlogs. Security operations teams rarely receive meaningful telemetry from build systems. The result is a visibility gap where malicious code can run, disappear, and leave organizations unsure what was touched or stolen.
The episode also explores why simple advice like “only use vetted packages” fails in practice. Open source ecosystems move too fast for manual approval models, and internal package repositories often collapse under friction. Meanwhile, attackers exploit maintainer accounts, typosquatting domains, and ecosystem trust to reach billions of downstream installations in a single event.
This discussion challenges security leaders to rethink how software supply chain risk is defined, detected, and owned. The problem is no longer theoretical, and it no longer lives only in development teams. It sits at the intersection of intellectual property, identity, and delivery velocity, demanding attention from anyone responsible for protecting modern software-driven organizations.
⬥GUEST⬥
Paul McCarty, NPM Hacker and Software Supply Chain Researcher | On LinkedIn: https://www.linkedin.com/in/mccartypaul/
⬥HOST⬥
Sean Martin, Co-Founder at ITSPmagazine and Host of Redefining CyberSecurity Podcast | On LinkedIn: https://www.linkedin.com/in/imsmartin/ | Website: https://www.seanmartin.com
⬥RESOURCES⬥
LinkedIn Post: https://www.linkedin.com/posts/mccartypaul_i-want-to-introduce-you-to-my-latest-project-activity-7396297753196363776-1N-T
Open Source Malware Database: https://opensourcemalware.com
OpenSSF Scorecard Project: https://securityscorecards.dev
⬥ADDITIONAL INFORMATION⬥
✨ More Redefining CyberSecurity Podcast:
🎧 https://www.seanmartin.com/redefining-cybersecurity-podcast
Redefining CyberSecurity Podcast on YouTube:
📺 https://www.youtube.com/playlist?list=PLnYu0psdcllS9aVGdiakVss9u7xgYDKYq
📝 The Future of Cybersecurity Newsletter: https://www.linkedin.com/newsletters/7108625890296614912/
Contact Sean Martin to request to be a guest on an episode of Redefining CyberSecurity: https://www.seanmartin.com/contact
⬥KEYWORDS⬥
paul mccarty, sean martin, software, supplychain, appsec, npm, javascript, ci, malware, opensource, redefining cybersecurity, cybersecurity podcast, redefining cybersecurity podcast
[00:00:00]
[00:00:36] Sean Martin: And hello everybody. You're very welcome to a new episode of Redefining Cybersecurity here on ITSP Magazine. I'm Sean Martin, your host, where if you know me, I, uh, have the honor and privilege of chatting with some really cool people about some really cool topics. And, uh, today is no exception to that. Mr.
Paul McCarthy is on with me. How are you Paul?
[00:00:56] Paul McCarty: Yeah. Good mate. How about you?
[00:00:57] Sean Martin: Doing? Doing great. Good. Good [00:01:00] to see you. We, we had a, a catch up the other day and I was like. We need to record something. Let, let's have a chat. And you said MPM and I'm like, let's do it.
[00:01:10] Paul McCarty: And you know, you know, I'm not shy about being around the microphone, so here we're
[00:01:13] Sean Martin: know, I know. That's why I love having you on and, uh, didn't get to see you at Cyber Con.
Um, but uh, hopefully I'll get down and, uh, and catch you in, in Queensland at some point and, uh, maybe do some diving or surfing while I'm there. Or maybe, I don't know, maybe I'll play some guitar with you as
[00:01:31] Paul McCarty: Yeah, let's do it man. Come over to my house and we can hang out here in the backyard. I'm actually, my desk is actually outside, like literally I'm outside. I got a cover over it, but I'm outside, so it's a, it's a tough life mate.
[00:01:43] Sean Martin: It is, it is a tough life. It's a tough life. I'd like to, uh, experience that toughness with you at some point. Um, all right, well, we'll, we'll make that happen. Uh, we'll play some music. Um, in the meantime, let's talk about, uh, app security package Security, NPM stuff and [00:02:00] all the surroundings. We're gonna, we're gonna speak to the CISOs and security leaders and AppSec.
Practitioners today. Um, before we do that, Paul, for those who don't know you yet, maybe a few words about what, uh, what you've been up to, what you're up to now, and uh, yeah, that'd be good.
[00:02:15] Paul McCarty: Thanks, man. Um, yeah, I've been in, I've been in this cyber game for a long time. Been in it over 30 years now. Um, and, um, I find myself in Australia. I've been here for almost 10 years. Um, I started as, uh, an A SPM company before A SPM existed as a Gartner Quadrant. I started, um, an application security company in 2018 called Secure Stack, um, which was great.
Um, you know, raise some money, build a good product, but ultimately was not, you know, didn't bring me into my, my Hawaiian, Queensland, you know, laying in the Sun Dream. So, um, here we are. I'm, I'm currently working for a great startup based out of, um, Vancouver, bc um, named Safety. I. But I specialize in, uh, offensive security in the software supply chain space.
Uh, [00:03:00] although a lot of my time is spent obviously identifying threats. So, um, and there's not a ton of work in offensive security and software supply chain, but, um, uh, you know, I kind of bounce back and forth between offensive and defensive. Um, and I specialize particularly in NPM an NPM, um, malicious package in what I call open source malware.
Um, that's kind of my, my very narrow specialty.
[00:03:24] Sean Martin: Yep. Yeah, it's a, it's a niche, but much bigger than many might think, even though it's very particular, very
[00:03:32] Paul McCarty: No.
[00:03:33] Sean Martin: Um, so I'm gonna assume a lot of, a lot of my listeners have some idea what you, what you just rambled off there. Um, but, but let's, uh, let's package it up a a little nicely for them. I'm gonna paint the world of.
Application security and delivery and the pipeline and, and where NPM fits in inside there. And then, then we'll get into what, what you're [00:04:00] seeing in terms of, of risks and attacks and, and, uh, exposure and all that stuff.
[00:04:06] Paul McCarty: Sure sounds good. Uh. For your listeners. I think the important thing is that, you know, we're all building product. Um, and that product is, um, you know, obviously, uh, you know, a lot of it uses JavaScript. Now JavaScript is NPM, the Registry, which is owned by Microsoft, it was bought in 2020, um, is the world's largest registry by several orders of magnitude.
It's bigger than anything else out there. Um, uh, it hosts about 3.7 million. Active packages. Um, there's over, uh, 1200 new packages added every day. There's 145,000 modifications to packages every day. It's just a huge thing. And the, the applications we're building, um, particularly with JavaScript has attack surface both on the developer's laptop in the sense [00:05:00] that JavaScript.
Typically runs in a browser. Um, and so bad guys are, are crafting, um, attacks, uh, that affect the developers that work for their organization locally on their machines, uh, but then also in the CI pipelines. Um, and JavaScript can run through Node as a service side technology too as well. So it's a very flexible, um, language to use, which is why bad guys use it.
And that's why 98.5% of malicious packages are stored in one registry. NPM. Five. It's
[00:05:31] Sean Martin: Wow. Wow. And we're working toward 99 a year.
[00:05:35] Paul McCarty: Yeah. We gonna get there this year.
[00:05:38] Sean Martin: Ah, geez. All right. So, all right. So what, what are some example, I mean, there's no, no lack of, uh, examples to pick from, I'm sure, but what are some of the things that you see common JavaScript wise, that, that sits in there that forget about the, the vulnerable stuff? It is just look at common everyday applications that people [00:06:00] are building.
They're pulling, pulling, uh, pieces from MPM for, to build their things.
[00:06:05] Paul McCarty: Yeah, I mean, JavaScript has become the defacto language, you know, for for front end and for many backend applications, and so it's ubiquitous in every organization. So your listeners, while they might not know if they're running. JavaScript and pulling from NPM lots and lots of packages. The reality is they probably are statistically speaking, um, and JavaScript in particular, um, not only is, like I said earlier, almost 99% of the malicious packages are in that space.
But JavaScript is much more, um, incestuous or it's, it's the number of dependencies than an average JavaScript application uses is much higher, um, than any other language by far. So, um, uh, there's a study.
[00:06:45] Sean Martin: service and call is just nested and nested.
[00:06:48] Paul McCarty: Well, yeah. Basically what it does is it calls a dependency, it calls a dependency, and each one of those has, you know, dozens or hundreds of dependencies. So, for example, um, in 2020, uh, GitHub did a, uh, a study and they found that the [00:07:00] average JavaScript project on GitHub. Had 683 indirect or transit of dependencies, that number is now over 1200.
Um, now back in 2020, number two was, um, was, uh, Python, I think with something like 60 dependencies, right? So it's just like 10 x the number two language, uh, JavaScript is just, you know, it's, it's built on this kind of, no batteries included. Philosophy where you need to pull in all these tiny little things to do everything, and then that just increases the attack surface.
Um, and NPM in particular, even though it's owned by Microsoft, it was a mosquito here, sorry, but it's, it's owned by Microsoft is, um, is actually unfortunately falling behind in terms of, you know, security and, and helping the developer protect themselves relative to other. Um, package ecosystems like the Pi Pi, the Python, the main Python one, one, which is called the Python, sorry, PI pi.
Um, they're [00:08:00] being much more proactive about it. Uh, now that's changed a little bit because in the last two months there has been NPM Argen, right? Like they're just, you know, you know, billions and billions of downloads have been affected over the last month and a half by attacks on some very popular maintainers, uh, which is.
It was that kind of milestone that we needed. It was kind of that unfortunate event that we needed to kind of raise the visibility for the average size O. And so we just see a lot more interest now from those security leaders asking about this, like, what are we doing? And the answer unfortunately is frequently, uh, nothing.
[00:08:38] Sean Martin: Or just, uh.
[00:08:39] Paul McCarty: Yeah, yeah, yeah. Like I give him a little more credit. It's usually just.
[00:08:47] Sean Martin: Um, yeah. So, so to, well, let's, let's not go straight to, uh, to what we're doing. What, what are, talk to me a bit about the, the injection [00:09:00] into the pipeline, which then compromises delivery and then ultimately run runtime. So what, what are some of the things you're, you're seeing in that, that space?
[00:09:09] Paul McCarty: Yeah, I mean that, um, that attack surface, the CI and the continuous integration and continuous deployment attack surface is huge and it's wide and there's a lot of great researchers doing stuff in this space. Um, um, and uh, it's got a lot more visibility now than it had a couple years ago. But the reality is it's the wild West.
So when you're building applications on your laptop, at least there's this opportunity for you to kind of. Observe and, and you know, the developer to, to, um. To analyze from a security perspective, you know what, they're running there locally. Most CI pipelines just run in total isolation. Um, they pull, you know, hundreds or thousands of packages, um, really quickly put those together, build an application and deploy it.
Um, and then they go away. You know, they're, they're transitive, they're, um, they go away. [00:10:00] So, um, a transient, sorry, transient, they go away. Um. And the problem is I think that we think that we have a lot more visibility in that space, but we, we just don't. Um. GitHub's doing a good job and GitLab's doing a good job of trying to to, to build some native tools into their platforms.
But the reality is that, um, when an attack happens in the CI pipeline, you're in this kind of really kind of crazy space. Sean, you're like, you're at this weird intersection of. A company's core intellectual property like these, these are the crown jewels, right? This is the code that runs your applications.
You sell this to your customers, right? Your customers run this on their machines and their servers and whatnot. You're at this intersection of that, this core intellectual property, and also more API keys and access than any other place. The density of access in a CI pipeline is unlike any other place inside your organization.
So those two coming in contact means it's just like the perfect place. For bad guys going look and they absolutely are. The amount of attacks that's happening in that space is just [00:11:00] skyrocket. I was looking at one yesterday. You know the first thing I did is it figured out. Which CI pipeline am I running in?
Is it Jenkins? Is it GitLab? Is it GitHub? And then based on that, it just went and ganged every single environment variable, every artifact it could, and, and, you know, sent it out to their endpoint. And that's happening in thousands and thousands of CI pipelines, you know, today. Um, and not a lot of visibility on, on when that's happening, and that's therein lies the problem.
[00:11:28] Sean Martin: And when, when is it happening? Is it, is it at build time or is it just as codes being written? Run? Is it compile, build? I don't know. When, when is it happening?
[00:11:39] Paul McCarty: I mean both. Um, a lot of the attacks now are focusing on the, the very, very left. The bad guys unfortunately have shifted left. We might not as application security pro providers, but the bad guys, man, I just think we coined something there. Right, right there, Sean. The, the bad guys have shifted less, left in a much more productive, efficient way than we have.
Um, so for example, [00:12:00] you know the Lazarus group, north Koreans, they are absolutely hammering developers. To, um, initially it was to, to steal personal crypt crypto that those develop developers might have. Right. And they're doing that through really kind of unique, innovative ways, like this contagious, um, uh, uh, interview campaign where they, you know, they, they.
They hit people up on, on LinkedIn and say, Hey, do you want this high paying job in crypto? And, you know, just download this repo. And there's just so much innovation going on in that space. So they're really focusing that contagious. Uh, interview campaigns, focusing on developers, on their laptops. What happens is you download some code, you run it, it imports some malicious packages and code, and then they can steal, you know.
Cookies, tokens, session tokens and crypto directly outta that. Um, as well as files local on the machine. At the same time, Lazarus Group is also attacking enterprise organizations with a whole other set of campaigns. Um, [00:13:00] so using, and this allows them to use the same kind of core, um, payload and delivery mechanisms for very, very different campaign focuses and targeting very, very different places.
So. They'll take some of that and they'll point it at the CI pipelines. Um, and so this is running in GitHub and GitLab and Jenkins and all kinds of other places build kit, what have you. But then they're also having different targeting on the developer's laptop. So, um, you know, being able to deliver security tools that cover off both of those places is difficult.
'cause you know, the attacks look different and they happen differently. So, um. It's a difficult thing for people to address, but um, you know, there's a lot of people talking about it. No sizes or
[00:13:45] Sean Martin: Right.
[00:13:46] Paul McCarty: I think suddenly clued in like, oh shit, this is a real thing.
[00:13:50] Sean Martin: And yeah, I don't know how much of, uh, the, the signals coming, even if they can. Put some monitoring on these things. What [00:14:00] signals end up in their, in their sim and in their sock.
[00:14:04] Paul McCarty: Yeah. I mean, there, yeah. The, in my experience, there's not a lot of, um, uh, you know, end-to-end telemetry and um, uh, you know, logging and whatnot that goes from the CI pipeline all the way into those security tools. So it doesn't happen that often. Are there mature organizations out there that are doing some of that?
For sure, but they tend to be separate kind of single panes of glass. Right. And so there, so there's kinda, the problem is that the AppSec team will have. Some visibility into the, to the software delivery part of it. But then the kind of important, um, you know, uh, threat intel stuff isn't moving into InfoSec, stuff that InfoSec owns.
Um, you know, obviously, like I said, there's, there's mature organizations out there that are doing that, but by and large it's, it's a very nacent space. Um, it's, it's very, very early and I don't see a lot of people that are moving that, you know, that data from. [00:15:00] Logs and what have you and events into security tooling.
[00:15:05] Sean Martin: Right. So it's left if, uh, if they're fortunate enough to have one left to the AppSec team. Otherwise it's left to, left to dev, um, and perhaps even an outsource dev team, right? That doesn't even.
[00:15:21] Paul McCarty: Yeah.
[00:15:21] Sean Martin: the business, but, uh, um, how, how do, uh, well, yeah, how do organizations, let's assume it's not going to, to SecOps.
Um, let's assume there's an AppSec team. What, what, uh, what, what are they doing to kind of get a view into what's going on here?
[00:15:41] Paul McCarty: Yeah. Well, I mean, I think the problem is the AppSec teams are typically not focused on this, right? And that's coming down from, from management, right? So that most AppSec teams are just. Almost singularly focused on this grind of burning down the backlog of vulnerability data, you know, of vulnerabilities coming out of their, their, their [00:16:00] tooling.
Um, and, and so they don't have any time or capacity and the team's not large enough. You know, the number of AppSec teams that are a team of one, you know, especially here in Australia, um, it's very, very common to have an ABSEC team of just one person. So unfortunately the AppSec team is not. You know, typically, um, you know, it doesn't have the capacity or the ability to, to be able to, um, to look at this kind of malicious attack stuff.
So then the problem is it falls on the InfoSec team, but the InfoSec team doesn't have any of the tools in the space. Now, here's where the problem gets even worse. It's compounded by the fact that the databases that we rely on, osv.dev and GitHub Security Advisory for a lot of this data are just not.
Fit for purpose. They're not built to handle malicious attacks like they, they, they were built in an earlier time when we only observed one software risk, which is. Accidental vulnerabilities, right? Which is a developer accidentally, you know, SQL injection and input, validation, all that kind of stuff, A wasp top 10 stuff.[00:17:00]
It wasn't until later we realized, oh, people are actually attacking us. They're, they're criminals, purposely malicious, has a totally different look and feel, but the databases themselves, um, and, and the infrastructure and the A SPM tools. Aren't, are typically not set up to pull that stuff in the right way.
And so you have situations where, for example, some of those big attacks that have happened in the last two months because GitHub didn't apply. A tag to a particular event that said malware. This is a crazy story, but it's true. They, it, it came in as a CVE and so it, like they were treating it like it was a vulnerability.
It didn't have this malware tag. So the problem is some of the A SPM tools, and I won't name any names, but it was more than one. They pulled the data from GHSA, GitHub Security Advisor and OSV, and they look for that malware tag to be able to highlight it inside the platform while the malware tag wasn't there.
What happened is for a week and a day, eight days, one of these platforms was not telling their [00:18:00] customers about the fact that these dozens and dozens of of packages were known malicious bad stuff like this is like, you know, these, these are bad packages with bad payloads. Were not telling their customers about it.
Because of this kind of fundamental flaw in the way that we kind of address malicious, uh, packaged software risk, we're just not set up to do it very well right now.
[00:18:23] Sean Martin: Right. Uh, so many things flying into my head now. Um, uh, uh, one is zero, trust. One is the, the whole, uh. AI coding and vibe coding and which I'm sure just adds complexity. I don't know. Let's go with, well, let's talk about that. What? I wanna go back to the engineers 'cause, so it's gonna fall onto them, uh, right when they're picking stuff for now, right?
Until we can, we can do something more, uh, further downstream. So how, how, how can they determine. [00:19:00] What's good, what's bad, what's malicious? What, uh, is there any, are there any signs or signals that
[00:19:08] Paul McCarty: Great question.
[00:19:09] Sean Martin: or attributes or, I don't know, whatever.
[00:19:11] Paul McCarty: Yeah, I mean, I think, you know, the company I'm working for safety, you know, they're, they're working on, on building this, kind of highlighting this to, to their customers. Uh, and one way to do that is to run something called a, um, a software firewall or a, uh, you know, source code firewall on the local developer's machine, which then.
You can kind of think about it like EDR for the developer, where it's like got this bi-directional relationship with the mothership, the mothership saying, Hey, this is the list of bad packages. Don't install these. If you see somebody installing these, raise an alert, it goes back to the mothership and then there's a, there's a notification, right?
Um, but the reality is that, you know, I think we're a long way away from organizations, you know, implementing those solutions right away. I think it's where it's gonna go. Um, but ultimately, you know, getting back to that earlier, go ahead.
[00:19:57] Sean Martin: I was gonna say the, the, the challenge there is, [00:20:00] um, as with most, uh, I'll put that in the bucket of zero trust, right? You only, you only use what you know is good. Um, you're gonna break a lot of stuff
[00:20:09] Paul McCarty: Yeah, you are.
[00:20:10] Sean Martin: and you want, you actually wanna deliver.
[00:20:13] Paul McCarty: So InfoSec for years has been saying, well just use vetted packages. And I'm sorry that just like that just shows how little they know about how the software sausage is made and delivered. Right. And that's a big problem. That's another problem, right? Which is that, you know, zooming out here to 40,000 feet, right?
InfoSec has not done a good enough job of trying to join the DevSecOps fray and understand how this stuff kind of natively works, right? So they make these blanket, you know, observations, like, just vet your packages, just pin your packages, right? And that's like, no, I'm sorry, that's not, that's not, that's not a big boy answer, right?
Um, the managing an internal. Package repository with something like Artifactory or whatnot, um, has its own challenges because developers will say, Hey, I need [00:21:00] this new version of that. And the InfoSec team says, okay, well we gotta take a look at that. We gotta, we gotta vet it before we add it to the thing.
Well, then there's a backlog while you wait for somebody to, to do that. And the team that's doing that, they might not be the best poised people inside that organization to make that determination, but yet that's where the task lies, right? So. What I've seen a lot, Sean, is that that organizations will implement these kind of internal package, um, ecosystems, and then very quickly it'll get overrun, get backlogged, and they just, they just ignore it and they go back to pulling stuff directly from the internet.
Which by the way, the reason that happens is just because of the, the frequency and iteration of change inside of, you know, these, these libraries and these packages. Um, you know, we want to be pulling new code. Right. So it's, it's addressing existing historical vulnerabilities in, in the old versions, but at the same time, if you run latest, you know, you, you're subject to this very small, rare situation where.[00:22:00]
A maintainer is, is compromised and they push out, the bad guys push out this, which is what's happened over the last two months, right? These, these big, huge events have all been about account takeover of, of these package maintainers. But the reality is that doesn't happen that often. So I think all things being said, it's more, and, and it's not black or white, it's not one or the other.
But the reality is that most organizations have this kind of compromise where with some things they let it go. You know, they pull the new most recent version of it and for certain things they pin it. And then somebody's gotta manage the pinning. How often are you updating that? Right. So there's a lot more work that goes into it.
And if the number of times I heard InfoSec just say, you know, we'll just, you know, vet the packages. If I had a dollar for every time, man, I'd have at least a case of beer mate. Seriously.
[00:22:45] Sean Martin: There you go. A case of beer. That's good. What, um, maybe describe the story of, of the maintainers getting popped and, and what that, what that means and looks like. Um. I mean, [00:23:00] for open source, for it to work, you want people looking at it, right? It's a community driven thing. And if you, you put all the onus on one person and it's not their job, but, but they're maintaining.
I dunno. So describe the story and maybe just the, that whole world of open source and the,
[00:23:18] Paul McCarty: Yeah, that's a great point there. I wanna touch on that last one first, which is that I think a lot of organizations, you know, I've, I've heard stories about enterprise, large, huge multinational organizations sending these letters to single maintainers saying, you know, are you soc to compliant? You know, fill out this questionnaire.
And it's like someone, this is somebody working on their spare time delivering a core functionality to the whole fricking internet, you know? It's like that, you know, it's like that famous meme where they're like one little piece, like, you know, they're doing their, they're maintaining that piece. These are, these are people with other jobs. Yeah. At the same time, I think that, you know, some of the responsibility is on those maintainers to, to stay [00:24:00] up to date and to implement. For example, um, you know, I use the Open SSF Scorecard app, which is a great open source tool. And basically what it does is it goes and guide, takes a look at an open source project and gives you the list, like baseline score and looking at all kinds of stuff, like how often is it maintained, are they scanning it?
And the reality is a lot of these open source tools don't have, if you go and look at their GitHub. Actions or their, their GitLab pipelines or whatever, they're not actually scanning their source code with security tooling and which is really easy to see 'cause it's right there in the public. 'cause it's an open source thing.
So I think there's some maturity there that has to happen with the maintainers. But you know, these enterprise organizations, the presumption that they have to send these letters to people, it's just, if you like, why don't you get involved? Maybe send some money. Maybe you pay
[00:24:45] Sean Martin: of the lawyer, instead of the lawyer writing a letter,
[00:24:49] Paul McCarty: Right.
[00:24:50] Sean Martin: have an engineer help review the code or contribute to the code or whatever.
[00:24:53] Paul McCarty: The, the, the other problem is that frankly, NPM hasn't done a particularly good job of protecting the maintainers. [00:25:00] So for example, pi, PI, you know, four, I think it was four years ago, roughly speaking, implemented MFA and some other, you know, protections and NP M's getting around to it. Now, they're doing a bunch of stuff now because of these huge attacks we've had in the last two months.
But the reality is the NPM still, you know, like anybody can grab a namespace in NPM, like. If, let's just say that at Microsoft was available, which by the way, it's not, I could just log into NPM with a brand new account. You know, nobody knows who I am and I could grab at Microsoft, right? And you know, whereas over in Pi, PI you have to verify that you own the domain and you have an email address or something, some reason, you know, there's all these little things they do in other ecosystems and not that pi pi is perfect, but NP m's not doing any of that.
Um, and they're owned by the world's second or first largest organization, you know, computer company based on current stock price. I don't know. Um, and, and so if there's any organization that could be helping out here, it's Microsoft, but they're not. They, Microsoft has, lemme just be [00:26:00] very clear. Microsoft has been derelict in their, um.
The way they've treated NPM, they really have, it's really kind of disgusting to me. Um, so, um, spicy take by Paul Spicy, take number 47.
[00:26:14] Sean Martin: There you go. Well, we'll see what they have to say about that.
[00:26:18] Paul McCarty: Are they one of your sponsors?
[00:26:19] Sean Martin: no, no, no sponsors on this. Um, we can say whatever the heck we want. Now, let's, um, give an example of one of the attacks, and I don't know, I may not, not MPM specific, but I mean, SolarWinds was. Similar type of deal. Right.
[00:26:40] Paul McCarty: It was.
[00:26:41] Sean Martin: And, and sadly a few people, I mean, it was a bigger, bigger conversation than, than the package or the, or the, the, the build environment.
But, um, yeah, people were held accountable or, you know, so I guess what I'm, I'm describe a [00:27:00] scenario and who, and then kind connected that I'm trying to figure out. Well. It's this age old question of software who, who owns,
[00:27:08] Paul McCarty: Right.
[00:27:08] Sean Martin: owns the liability when something bad happens? I.
[00:27:12] Paul McCarty: Yeah. I mean, let me, let me answer that question with a story about pi. Pi, which, and by the way, PI. Pi has done a really good job over the last four or five years of really kind of. Uplifting their security programs. I mean, there's still some things they could do better, but, um, I really have to give Mike and his team some credit there.
But, um, so when they were implementing their, their multi-factor strategy, so basically, um, the important maintainers and, and, um, pipe, I, you know, got this email saying, Hey, by the way, we're gonna force you and all the other popular maintainers to implement MFA, but we're not gonna do it right away. We're gonna give you time to do it.
You know, so they send another email like a month and a half later, Hey, just letting you know that, you know, in four months we're gonna require you to do this. And they send another email. But then, then Sean, the bad guys. The bad guys. Basically, it's just really easy to figure out who all the top maintainers are in pii.
You just go and look at the [00:28:00] download numbers. Same thing with MPM. Super easy to find out what the bad guys did, and you can get the email addresses, the maintainers from the package metadata, right? So what the bad guys did is they just sent another email, but it wasn't from Pi Pot, it was from the Bad Guys.
And it said, Hey, you know how we told you three or four times already that you had to implement MFA? Well, today's the day. If you don't do it right now, you're gonna lose access to your account. And the number of people that clicked on that and that one. I mean, that was, there was nothing nuanced about that.
There was nothing special about it. It was just a bad guy using timing right. And context in a phishing email to be very, very effective at, at delivering their, their, um, their attack. So similar things happened in NPM, and that's a lot of what's happened over the last two months where basically maintainers get an email saying, Hey, you know, you just gotta.
Update something or just log in and do this thing. And, um, in one of those cases, the domain was npn js.com, not [00:29:00] npm js.com. It's like, why Microsoft didn't buy NPN js.com? I don't know. I'm guessing they own it now, but, um, you know, it doesn't, this is not rocket science. You just gotta do the right thing and just make it look good enough.
And, you know, you get one or two of these top maintainers and some of these maintainers. Their packages are downloaded billions of times a week, right? You only gotta pop one of those or two of those, and then you just have, and that's exactly what's happened over the last two.
[00:29:28] Sean Martin: Yeah. And what, what's the, what's the cleanup look like for that? Are we, are we talking like node or not node? Uh, what was it? Uh, yeah. No, the, oh boy, I can't, my brain's fried here. The, the, the big, the big patch that hit pretty much everybody,
[00:29:51] Paul McCarty: Oh, oh, singularity or shy ude or, or I don't know which pasture. SolarWinds.
[00:29:58] Sean Martin: Um, what, what, [00:30:00] yeah, so the environment, the packages compromised, get distributed. What's the cleanup look like for that?
[00:30:07] Paul McCarty: Yeah. I mean, listen, that's, that's one place where NPM typically moves really quickly, you know, so they'll, they'll go and find the, the malicious versions and they'll delete them outta the repository. Which, by the way, I've written a NPM package that'll lets you pull those deleted ones outta the registry.
It's called ele. You can find it on NPM. But anyhow, um, the, the, you know, they do a good job of removing it. And so the great thing then is that if you try, if your CI pipeline tries to pull that, again, it won't, unless you're pulling from some weird cached or, or, you know, third tier. Um, CDN, but the reality is the damage is already done.
It's already run, right? So it's either run on your developer's laptop or it's run in your CI pipelines or both, and you, and now you have to move into incident response mode. But the reality is getting back to what we were talking about like 10 minutes ago, like the tooling and the kind of understanding and the context from the InfoSec teams about how you do InfoSec, sorry, how [00:31:00] you do incident response, you know, just.
They don't have a lot of history of doing that with this kind of stuff. So what are they looking for? What are you even looking for? What, what tools are gonna tell you? What the C two was for that payload, right? And that, and that, that gets back to those observations about, um, GitHub Security Advisory and OSV.
They're not capturing that data. So people don't know what to look when they go into incident response mode. They're literally, this is the messed up thing, Sean. There's literally nowhere to go and find out if an NPM package has been deleted and you can't go and inspect it yourself. There's nowhere to find out where this data is to do instant response.
Um, and that's one of the reasons that.
[00:31:36] Sean Martin: Wow.
[00:31:37] Paul McCarty: Yeah, well, I've been working on an open source, um, project that basically replaces OSV and G HSA called open source malware.com. It's, it's an open thing. Anybody can pull from it and that data is in there. Right? Um, uh, because frankly, when I'm doing incident response.
I needed it. And so I just went and built it because nobody else was, you know, the people that were supposed to be delivering this [00:32:00] wasn't. So we're starting to get some of these kind of tools in place. Things like open source malware.com and, you know, these, these source code firewalls and stuff, they're gonna help you with incident response.
'cause for example, like safety is firewall. When, you know, when you install something, there's a, a message sent back to the, to the centralized, um, uh. Uh, platform that says, Hey, you know, Sean installed this version on this date. So then when you go later on and find out, oh shit, that was malicious, right? You find out after the fact, you can go back and say, okay, it was installed on this and this machine at this date, and now you know where to go and look for it.
But I mean, these tools are all new. We didn't have this stuff a year ago, so we're starting to get there. Um, but instrument response is still, you know, tough in these situations if you don't have some of those tools in place.
[00:32:48] Sean Martin: All right, so I was trying to find an episode where I talked about the other thing that's slipped my mind, but I'll. So we have this [00:33:00] environment and I don't know, here's what I wanted to, we can, I don't know if it's related directly to, to zero trust or not, or flagging good packages versus bad, but there's been a lot of talk about SBOs. What, what, what are your thoughts on that? How, how does that play a role in this?
[00:33:21] Paul McCarty: Yeah, I mean, oh gosh. This, this could be a whole nother episode, mate. Um, li listen, like, I mean, at a high level, SBOs are su super important. You know, if you're delivering, um, software to your customers, you probably should be building an Sbo m and SBOs if they're built and stored right. It can be a very powerful thing in an incident response, or even in proactive, you know, proactive security purposes.
Um, but the reality is most people don't create their SBOs like that. Like, and you know, first at Secure Stack, we, I came up with this whole idea of this lifecycle of SBOs, which is you do discover, you find out where all your stuff is, then you generate the [00:34:00] SBOs, then you store them somewhere together, and then you make them searchable.
So the next time there's like, log for J Is that the one you were thinking about is Log for J. Nailed it. You know, the next time there's, next time this log.
[00:34:12] Sean Martin: kept saying no to my head, but log four J, that's.
[00:34:14] Paul McCarty: As soon as I said that, I was like, I bet that's what Sean was talking about. You know, when there's the next log for J log for log five J, whatever, um, you know, you have one place to go and gr and say, and search and say, Hey, where do I have this package?
Right. Um, so SBOs can be, you know, very, very powerful things. Before and after in an incident. The problem is that InfoSec again has kind of latched onto this, you know, because it's, you know, the executive, um, action in the US and you know, all this focus on it over the last six or seven years in the US, InfoSec has latched onto it, but they don't really understand.
How to do it the right way. And so they think it's like a pen test report, like you do one and then you've got it. Like, Hey, here's my sbo, right, here's my sbo, you want my pen test? You want my sbo? [00:35:00] And that's not how it works. You should be generating an SBO every single time you build your application, whether it's in dev staging or prod.
There should be an SBO m each one of those places. Um, especially if those things are, um, unfortunately dev and staging are now often public, whereas, you know, 10, 15 years ago they weren't. But anyhow, um, you know, generating those.
[00:35:19] Sean Martin: blending. Blending, and, uh,
[00:35:22] Paul McCarty: I mean the cloud, you know, with the cloud came public facing dev environments, right?
Like, um, but yeah, I mean, you know, having that, having those SBOs and being able to search them but making sure they're comprehensive. And that's the other problem, you know, not all SBOs are, you know, not all pizza Sean is, is equal, right? There's shitty pizza and there's good pizza.
[00:35:44] Sean Martin: Yes, there's a lot of good pizza in New York, thankfully.
[00:35:48] Paul McCarty: There is, there is.
[00:35:50] Sean Martin: so I'm glad we talk about pizza. I love talking about pizza. Let's, um. We're, we're at 35 minutes here. Uh, we could probably talk for hours more, [00:36:00] but let's kind of, let's bring it to, to a wrap here. And I wanna speak to who do we wanna speak to, security leaders or AppSec teams?
Who, who, who, who could we say something to right now that will make a difference for them and when they wake up and go to work tomorrow?
[00:36:18] Paul McCarty: Yeah, I mean, let's, let's talk to the leaders, right? I mean, you know, we're in this position where, you know, we, we have this threat. I've known about this threat for a while, but suddenly this threat, now we've hit this kind of, you know, inflection point where now I think CISOs and security leaders see this as a real thing, and that's great.
Understanding it, being able to visibly define it, um, doesn't mean they necessarily understand it completely, but they see it as a threat. And that's good. Where do we go from here, Sean? Like, what do we do? Um, and the reality is that, you know, we do need to buy some tools. We need to, to do some, some training.
Um, but we need to create some automation around the way that we [00:37:00] build and distribute our software so that we start to tackle this problem. If we keep saying, you know. This thing or that thing is gonna fix it for us? No, you gotta, you gotta get involved, right? Um, none of these ASM A SPM platforms are just gonna do it automatically for you.
Well, most of them aren't. Um, so we need to be proactive. We need to get engineers involved. Um, and we need to help them understand that, you know, bad guys are targeting your, your individual laptop. They're targeting the dev that you're running locally on your machine. That means we need to do something about that.
[00:37:34] Sean Martin: Right. Okay. Uh, and I suspect they need to talk to you 'cause I, I can't help them.
Hopefully there's smart people that know, uh, know how to take a look at that environment and, and start to put some, some protections, at least detections in place and, and from there, protection, hopefully. Um, yep. Paul, it's always good to see you, my friend.[00:38:00]
[00:38:00] Paul McCarty: Indeed. Did you?
[00:38:01] Sean Martin: thanks for, thanks for having this chat with me and, uh, scaring my pants off and, and, and making me hungry for pizza.
All, all in the same, all in the same episode.
[00:38:12] Paul McCarty: I like it. I like it. Maybe, maybe pizzas in my, uh, near for lunch mate. Here we.
[00:38:16] Sean Martin: There you go. It could be, could be. All right. Well, uh, yeah, I encourage everybody to, uh, connect with you. You're a super smart dude and super connected in the community, and, uh, it's always, always a pleasure chatting with you and seeing you. And hopefully I'll get down in, uh, get down to, to, uh, Australia, Queensland, and, uh, and we'll enjoy the, enjoy the Golden Coast.
[00:38:40] Paul McCarty: I'll be not too far away from you. I'll be in DC all next week, so, um,
[00:38:44] Sean Martin: There you go. I don't know if I'll get to DC Oh, that's right.
[00:38:47] Paul McCarty: Yeah it is. Yeah. I'll be there doing my thing in chicken wing.
[00:38:50] Sean Martin: not gonna, I'm not gonna make it to sadly. Um, yeah, I know. I'm, I'm stuck here working, stuck here working. Anyway, [00:39:00] Paul, it's good to see you, everybody listening to, uh, and watching this episode. Appreciate you joining me here.
And, uh, stay tuned for more. I have a lot coming up. A lot of episodes I'm recording and, uh, check out the newsletter that I'm writing until I have a seven part series on AppSec that I'm working on. And the last one I just wrote touched on this very briefly, um, kind of the, the vulnerabilities in the pipeline and.
And kind of to the point we just said, we're not thinking about it too much. So, uh, yeah, we have to start thinking about it. All right. Stay tuned. Uh, future of Cybersecurity Newsletter and, uh, redefining Cybersecurity podcast. Thanks everybody.
​[00:40:00]