About Us | Contact Us
View Cart

When New Technology Brings New Risk

By Vigilize | Monday, May 17, 2021 - Leave a Comment

Machine learning is here to stay, so how do we assess its risk?


An article review.


When it comes to assessing technology risk, there seems to be as many methods as there are attack vectors… but what happens when an entirely new field opens up?  When it comes to machine learning (ML) there aren’t many methodologies available, but a team of Microsoft employees are working on changing that.  The team in question is working on Counterfeit, a tool for assessing the security of ML systems, and in addition to creating this command-line tool they have written some interesting guidance on how you can begin to assess ML risk on your own.

The guidance includes a brief description of the ML development lifecycle, which begins with collecting the data which will be used to train the model and continues through processing that data, feeding it to the ML algorithm, validating its output and finally putting the system into production.  Once the system is in production the job isn’t over, though: ML algorithms must be continuously monitored to ensure their output remains valid.

In addition to needing to ensure the ML system is producing valid output, a number of attacks exist that could reveal the data the model was trained with.  If the model in question was trained with personally identifiable information (PII), this could raise questions regarding compliance with standards such as HIPPA or GLBA.

Finally, the guidance looks at the methodology behind potential ML attacks.  Starting with information gathering about the application, an attacker may then focus on attacking the ML algorithm itself, either to reveal data or to repurpose the algorithm to perform a potentially malicious task.  It’s also important to realize that while ML is new, it is still a component of your existing technology infrastructure.  Therefore all the traditional technology risks are also present, and a compromised workstation or server could also lead to a potential compromise of the ML system.

Machine learning offers tantalizing potential benefits, but as it is a rapidly developing technology one must also stay ahead of the curve when it comes to new threats.  Luckily frameworks like the one provided by the Azure team are being developed to assess those risks, and it certainly won’t be the last.


Original article by the Microsoft Azure Team.


same_strip_012513


 

Posted in Article Reviews

Latest News
    A follow-up on Dan’s 2008 Password Manifesto On the NIST Publication on Digital Identity Guidelines Another one of those Dan’s New Leaf Posts, meant to inspire thought about IT Governance . . . . In June 2017, NIST released a special publication on digital identity, NIST SP 800-63, that is starting to get the attention […]
    Another awareness poster for YOUR customers (and users).  Now that we have our own employees aware, maybe it’s time to start posting content for our customers! Download the large versions here: Awareness Poster (Portrait) Awareness Poster (Landscape)   You are welcome to print out and distribute this around your office. Interested in one of ours […]
    Over Seven Billion Usernames Have Been Leaked in Breaches Since 2011… An article review. An unfortunate fact of modern life seems to be the inevitable announcement of new data breaches, and if you’ve lost track of how many breaches you’ve had to perform a risk assessment on you’re probably not alone…but just how much personal […]
    Or, the risk of email hypnosis . . . And the other implications of complacency! Another one of those Dan’s New Leaf Posts, meant to inspire thought about IT Governance . . . . Now that the pandemic is coming to an end, most of us are returning to our daily commutes.  Are you finding […]
    Another awareness poster for YOUR customers (and users).  Now that we have our own employees aware, maybe it’s time to start posting content for our customers! Download the large versions here: Awareness Poster (Portrait) Awareness Poster (Landscape)   You are welcome to print out and distribute this around your office.  
    Machine learning is here to stay, so how do we assess its risk? An article review. When it comes to assessing technology risk, there seems to be as many methods as there are attack vectors… but what happens when an entirely new field opens up?  When it comes to machine learning (ML) there aren’t many […]
    You’ve heard it from every MSSP you’ve met: the definition of a SIEM is in the eye of the beholder. But at infotex, we are not talking about the database – an asset whose definition is continuously evolving. We’re talking about the way three teams collaborate in an overall Technology Risk Monitoring process. And whether […]
    A new study shows organizations are responding to cyber attacks faster than ever, so why is that bad news? An article review. When it comes to cyber attacks, the sooner an organization can begin to respond to an attack the better, so the results of a new study showing a drop in the amount of […]
    …a Crash Course of Security Measures The first article by Sara Fultz, Creative Assistant of infotex! Introduction: As the managing partner of infotex, I am proud to introduce the “debut article” for Sara Fultz.  I told Sara “write an article showing us what you’ve learned that the technical staff will appreciate.” As I read her […]
    infotex Programming Coordinator, Michael Hartke, introduces a high level overview of the upcoming update to the infotex SIEM. Look for more movies in the coming months informing our Clients, and those just now learning about us, about the SIEM and its features and functions.