Wednesday, January 31, 2024

Russian 677 Lada Submarine Kilo Replacement: a super stealth winner or poor looser?

Lada class sub brief - Old style design submarine

@SubBrief Aaron  Amick

We did a lot of research on this one because there's not a lot of public information on this lesser-known Russian submarine  

Our Story begins really in 1987. The cold war is really at its peak. This is right before the five-year or so decline to the end of the Soviet Union and they're just going all out with their spending coming up with new ideas. A lot of the money at the end of the Soviet Union was really put into rocket technology and missiles and things like that, but they also put a lot of money into the Navy and one of the new ideas was to get away from nuclear propulsion for our submarines because conventional is cheaper for one and we can get a lot of performance out of these diesel conventionally powered submarines and that spurred on this idea part of which was from the kilo.

The kilo class conventionally powered submarine was a major success for the Soviet navy. They built a ton of those boats and they improved to modernize them over and over again to where they were going to be very well. The kilo is absolutely famous for being an outstanding sub and this submarine the lot of submarine that nobody knows about is the replacement for the kilo

This is the new kilo. This is the new super stealth sub that is entering Russian Service as of this year and nobody knows about it, so that's what we're going to fix today. We're going to tell you about this new sub and what a hunk of junk it is. The reason why it's so secret is they're embarrassed to tell anybody about it. This is a complete train wreck from design to construction to sea trials that nearly killed people on board. It's a mess This submarine was designed by the very respected Ruben Central Design Bureau. Now they have a ton of successes under their belt. They've built every major nuclear submarine for the Russian Navy and a couple submarines that were not nuclear, as well designed them including the kilo class so they can design Subs. But something went terribly wrong with this one.

Designed around a unique implementation of air-independent propulsion (AIP) technology, the new Project 677 Lada submarines were meant to provide the Russian navy with a modernized, cost-efficient complement to nuclear-powered submarines. But the Lada project stalled amid technical difficulties, leading the manufacturer to abandon AIP propulsion altogether in favor of a traditional diesel-electric system. Project 677’s place on the looser list is a reflection of the fact that the Russian shipbuilding industry has failed to implement the submarines’ core defining feature, dooming what was a potentially innovative class to long-term technical irrelevance.

 

Russia's AIP-powered Lada-class submarines: A significant advancement in non-nuclear underwater warfare 

Russia's Lada-class submarines, also known as Project 677, utilize air-independent propulsion (AIP) technology, setting them apart as the first class of AIP-powered submarines in the Russian Navy. This technological leap allows for underwater operations without the need for atmospheric oxygen, enhancing their stealth capabilities and effectiveness in anti-submarine warfare. 

Improvements over the Kilo-class submarines include better acoustic signatures, advanced combat systems, a mono-hull design, increased speed, and reduced surface displacement for maneuverability. Lada-class submarines are equipped with modern sonar systems, automated combat control, and countermeasure electronic support. 

Despite initial plans to replace older Kilo-class submarines, construction of additional Improved Kilo-class boats has been ordered due to setbacks with the Lada-class program. Overall, the development and implementation of the Lada-class submarines represent a significant advancement in non-nuclear underwater warfare technologies.

Developer: Stealth Capabilities of 677 Lada Subs Outshine Its Predecessors

Visitors look at St Petersburg diesel electric submarine of the Lada class at the International Maritime Defense Show in St Petersburg. File photo - Sputnik International, 1920, 31.01.2024

The 677 Lada-class diesel-electric submarines are designed for reconnaissance, surveillance, anti-submarine warfare, anti-shipping, and mine-laying missions. The subs are equipped with advanced technology, including improved stealth capabilities, enhanced communication systems, and advanced torpedoes.

The stealth capabilities of the Project 677 Lada non-nuclear submarines are significantly superior to those of their predecessors, said Igor Vilnit, general director of the submarine's developer, the Rubin Central Design Bureau of Marine Engineering.

The first serial-produced Project 677 Lada submarine, the Kronstadt, built at the Admiralty shipyards in St. Petersburg, is being readied for delivery to the Russian Navy.

"In terms of stealth parameters, this submarine is several times superior to its predecessors. The boat maintains an extremely low noise level thanks to its specially designed equipment. Additionally, it extensively incorporates modern acoustic protection equipment, an external anti-hydrolocation coating, and carefully designed hull contours to ensure low visibility," Vilnit said in an interview.
"I would like to emphasize the hydroacoustics of the Lada: it is not only a wider range of acoustic waves, it is a significantly longer range of target detection. It took a lot of effort to achieve this result," Vilnit specified.
Mozhaisk Submarine Joins Russian Pacific Fleet - Sputnik International, 1920, 28.11.2023
Military
New Submarine Joins Russian Pacific Fleet

Lada-Class: Russia's Failed Diesel Submarine (No One Will Buy It)

by Peter Suciu Follow PeterSuciu on Twitter L

In July, the Project 677 Lada-class (NATO reporting name "St. Petersburg") diesel-electric submarine Kronstadt completed a deep-sea immersion as part of its continuing sea trials, within the maritime ranges of the Baltic Fleet.

The submarine crew verified the functioning of all onboard mechanisms and systems, practicing control algorithms at great depths and under various surfacing conditions, Naval Recognition reported at the time.

During the sea trials, the Kronstadt reached a depth of 180 meters at one of the Baltic Fleet's ranges, while the dive was facilitated by fleet forces and resources, including the rescue ship SS-750. The results of the submarine's dive were reported to the Commander-in-Chief of the Russian Navy, Nikolay Yevmenov.

Lada-Class: The Slow Boat

Construction of the Kronstadt – the first serial submarine of Project 677, following the Sankt Peterburg – began in July 2005 but was suspended by the Ministry of Defense of the Russian Federation in 2009 until 2013.

The submarine was only launched in 2018, and it took until the end of 2021 for the boat to commence its sea trials, which were prolonged due to ongoing modernization.

The submarine had been scheduled to join the Russian fleet this year, yet, there have been no reports that has occurred. When the Kronstadt finally enters service, it is expected to serve in the Northern Fleet, where the lead submarine of the project, Sankt Peterburg, is already in service. That decision had been announced at a conference call meeting at the Ministry of Defense in early March this year.

It was during that call that Russian Minister of Defense Sergey Shoigu confirmed that the submarine would carry Kalibr cruise missiles.

"The first issue on the agenda is the construction of the large submarine Kronshtadt that will enter service with the Northern Fleet. The ship is set to feature Kalibr cruise missiles, the latest radar, sonar and navigational systems. This will considerably boost its combat efficiency," Shoigu was quoted as saying by state media outlet Tass.

Lada-Class Key Details

Project 677 Lada-class submarines are often referred to as the fourth generation of diesel-electric submarines, developed by a Russian Rubin design bureau. It is essentially an improved version of the Kilo-class and was designed to be fitted with an air-independent propulsion (AIP) along with new combat systems. The AIP system was meant to increase submerged endurance to 45 days, while its submerged cruising range was 500 nautical miles (900 km) at three knots.

The boats have a surface displacement of about 1,750 tonnes and can develop an underwater speed of up to 21 knots, and an endurance of 45 days. The Lada-class subs are armed with Kalibr cruise missile systems along with six 533 mm torpedo tubes for a mix of 18 torpedoes or tube-launched missiles. These may include the Alfa (NATO reporting name SS-N-27 or Sizzler) multi-role cruise missiles, or Oniks (SS-N-26) anti-ship cruise missiles.

The boats reportedly have a crew of 35 including officers and sailors.

The boats were initially developed and designed to protect naval bases, coastal installations, and sea lanes from hostile submarines and ships, while these boats can also perform patrol and surveillance tasks, including anti-submarine warfare (ASW) and anti-surface warfare (AsuW) operations.

Problem-Plagued Platform

Sankt Petersburg (B-585), the lead submarine of the project, was launched in late 2004 and commissioned in 2010. However, she was not accepted by the Russian Navy as it was discovered there were issues with the boat's propulsion and that its sonar systems did not meet Russian specifications.

Construction on the remaining boats of Project 677 was thus frozen.

The issues with the lead submarine were eventually addressed, but only after several years of serving as a "test platform," she was formally accepted into service with the Russian Navy last year. Sankt Petersburg officially joined the Northern Fleet in September 2021.

Russia was unable to resolve the issues with the fuel cells for the AIP, and as a result the second boat of the class, the Kronstadt, was fitted with an ordinary diesel-electric propulsion system without the AIP system.

Currently, the Admiralty Shipyard is also building one more Lada-class submarine, the future Velikiye Luki, while the first steel was cut for the next two boats with a third also on order. Originally a full dozen of the diesel-electric boats were ordered, but given the issues with the program, it is unclear if that order has been pulled.

Export Model – With No Buyers

Russia had further developed an export variant of the Lada-class: the Project 1650 Amur-class (named for the Amur River), which was designed for markets including India and China, while Morocco has also been offered one. The export submarine could be offered in various configurations with a displacement of 550 to 1,850 tonnes and be equipped with a variety of weapon systems.

To date, there have been no buyers for the submarines, and given the problems with the boats as well as the sanctions placed on Russia, following its unprovoked invasion of Ukraine, the Amur-class could be dead in the water.

Author Experience and Expertise

Peter Suciu is a Michigan-based writer. He has contributed to more than four dozen magazines, newspapers, and websites with over 3,200 published pieces over a twenty-year career in journalism. He regularly writes about military hardware, firearms history, cybersecurity, politics, and international affairs. Peter is also a Contributing Writer for Forbes and Clearance Jobs. You can follow him on Twitter: @PeterSuciu.

Image Credit: Creative Commons. 

 

Lada Class Patrol Submarine | MilitaryToday.com

ARG

Country of origin Russia
Entered service 2010
Crew 38 men
Diving depth (operational) ~ 250 m
Diving depth (maximum) 300 m
Sea endurance 45 days
Dimensions and displacement
Length 72 m
Beam 7 m
Draught 6.5 m
Surfaced displacement 1 675 tons
Submerged displacement 2 700 tons
Propulsion and speed
Surfaced speed 10 knots
Submerged speed 20 knots
Diesel generators 2 x 3 499 hp
Electric motors 1 x 5 576 hp
Armament
Missiles Alfa (SS-N-27 Sizzler) cruise missile, Oniks (SS-N-26) anti-ship cruise missile
Torpedoes 6 x 533 mm torpedo tubes, for 18 torpedoes, anti-submarine or anti-ship missiles
Other mines in place of missiles and torpedoes

   The Project 677 or Lada class is a diesel-electric patrol submarine, developed by a Russian Rubin design bureau. It is an improved version of the Kilo class, fitted with an air-independent propulsion and new combat systems. The previous Kilo class was a basic submarine, simple in design and technology. Also it achieved respectable export sales. Its major operators are China, India and Iran. Development of the Lada class commenced in the early 1980s. It was rather protracted. The goal was to develop a submarine that would be much quieter than its predecessor. The lead boat was commissioned only in 2010. It turned out that the new boat has fallen far short of requirement.

   The lead boat was laid down at the Admiralty Shipyard in St. Petersburg in 1997 and launched in 2004. It was commissioned in 2010 and is in service with the Baltic Fleet. The Admiralty Shipyard laid down another three submarines of this class. The lead boat, Sankt Peterburg, was extensively tested by the Russian Navy, before entering service. Though it turned out that this submarine has fallen far short of requirements. The main problem was its propulsion system. Also there were a number of other major issues. Russia invested a lot of time and resources in development of the Lada class boats, however this project turned out to be a failure. One of the reasons was that after collapse of the Soviet Union a number of companies that produced various components for Soviet submarines simply closed down or stopped production of military equipment. Some of the companies ended up in independent Ukraine. So at that time Russians lacked all necessary equipment and expertise to build these new advanced boats. Between 2009-2011 construction of the follow-on boats was suspended due to multiple major issues with the lead boat. Rubin design bureau was ordered to make changes to the project. The Sankt Peterburg was used only as a test boat for testing various equipment, rather than for active duty. Incomplete boats were heavily redesigned and were built to an improved project. In 2013 construction of the second boat resumed. In 2015 construction of the third boat resumed - it was re-launched due to the redesign. However construction of the 4th boat remains suspended and there are no plans to complete it. Production of the Lada class boats was stopped in favor of more traditional Project 636 Varshavyanka (Improved Kilo) class boats. However in 2019 a contract was signed for construction of two more Lada class boats. Most likely that it included the 4th boat Petrozavodsk. In 2020 a 6th boat of the class was ordered. Interestingly the lead boat Sankt Peterburg began its active duty only in 2021. Originally it was planned that the Lada class boats will have a service life of 30 years.

   The Lada class submarines are designed to protect naval bases, costal installations and sea lanes from hostile submarines and ships. These boats can also perform patrol and surveillance tasks.

   The Lada class boats had a number of new and unusual design features. Designers abandoned a number of proven features in order to achieve better performance. These include new anti-sonar coating of the hull, which reduces acoustic signature of the boats. Submarines are fitted with sophisticated sonar equipment with bow and flank arrays, as well as towed array.

   The Lada class has six 533 mm torpedo tubes for a mix of 18 torpedoes or tube-launched missiles. These include Alfa (Western reporting name SS-N-27 or Sizzler) multi-role cruise missiles, or Oniks (SS-N-26) anti-ship cruise missiles.

   This submarine class is fitted with a fuel cell plant, which gives air independent propulsion with oxygen/hydrogen fuel cells and electric/chemical generators. The Air Independent Propulsion (AIP) system increases the Lada class submerged endurance to 45 days. The submerged cruising range is 500 nautical miles (900 km) at 3 knots. However it appeared that design of the Russian AIP was rather raw and had numerous problems. Notably the fuel cells were poor. Russia could not develop more advanced fuel cells due to funding problems and lack of expertise. As a result the second boat of the class, the Kronstadt, was fitted with an ordinary diesel-electric propulsion system without the AIP system. In 2022 the second boat of the class, Kronstadt, completed factory sea trials

Variants

   Amur class, or Project 1650, a less capable version, intended for export. It is named after the Amur river. Design work has been completed for a whole family of submarines with a displacement ranging from 550 to 1 850 tons and various weapon systems.

Name Laid down Launched Commissioned Status
Sankt Peterburg (B-585) 1997 2004 2010

Active, in service

Kronstadt (B-586) 2005 2018 Expected in 2022

Sea trials

Velikiye Luki (B-587) 2006 ? Expected in 2022

Under construction

Petrozavodsk 2006 ? ?

Construction suspended

Vologda 2022 ? ?

Under construction

Yaroslavl 2022 ? ?

Under construction

 

Tuesday, January 30, 2024

Evaluation of LLM Chatbots for OSINT-based Cyberthreat Awareness

Evaluation of LLM Chatbots for OSINT-based Cyberthreat Awareness

Computer Science > Cryptography and Security

Knowledge sharing about emerging threats is crucial in the rapidly advancing field of cybersecurity and forms the foundation of Cyber Threat Intelligence. In this context, Large Language Models are becoming increasingly significant in the field of cybersecurity, presenting a wide range of opportunities.

This study explores the capability of chatbots such as ChatGPT, GPT4all, Dolly,Stanford Alpaca, Alpaca-LoRA, and Falcon to identify cybersecurity-related text within Open Source Intelligence.

We assess the capabilities of existing chatbot models for Natural Language Processing tasks. We consider binary classification and Named Entity Recognition as tasks.

This study analyzes well-established data collected from Twitter, derived from previous research efforts. Regarding cybersecurity binary classification, Chatbot GPT-4 as a commercial model achieved an acceptable F1-score of 0.94, and the open-source GPT4all model achieved an F1-score of 0.90. However, concerning cybersecurity entity recognition, chatbot models have limitations and are less effective.

This study demonstrates the capability of these chatbots only for specific tasks, such as cybersecurity binary classification, while highlighting the need for further refinement in other tasks, such as Named Entity Recognition tasks.
Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Cite as: arXiv:2401.15127 [cs.CR]
  (or arXiv:2401.15127v1 [cs.CR] for this version)

Submission history

From: Samaneh Shafee [view email]
[v1] Fri, 26 Jan 2024 13:15:24 UTC (683 KB)

Summary

Here is a summary of the key points from the documents:

  • The documents present an evaluation of the capabilities of chatbots and large language models (LLMs) for natural language processing tasks in cyberthreat detection. Specifically, the research focuses on binary text classification and named entity recognition using Twitter data.
  • Several chatbot models are examined, including open source options like GPT4all, Dolly, Alpaca, and Falcon, as well as commercial versions of ChatGPT. Their performance is compared on classifying tweets as cybersecurity-related or not, and on identifying organization and product version entities.
  • For binary classification, ChatGPT and GPT4all achieve the best results, with F1 scores above 0.9. The other models range from 0.64 to 0.86 F1. ChatGPT is also most accurate for entity recognition. Overall, commercial models outperform open source, but GPT4all comes closest.
  • Challenges identified include inconsistencies in chatbot responses, requiring manual validation, and limitations in providing precise named entities without additional fine-tuning. Prompt engineering is noted as an important factor in optimizing chatbot performance.
  • The potential of LLMs and chatbots for cyberthreat detection is demonstrated, but refinement is still needed for optimal real-world application, particularly on specialized tasks like entity extraction. The study provides insights into strengths and weaknesses of different models.

F1 Scores

The F1 score is a common evaluation metric used to measure the accuracy of models for classification and information retrieval tasks. It is the harmonic mean of precision and recall:

F1 = 2 * (precision * recall) / (precision + recall)

Where:

  • Precision is the fraction of predicted positive cases that are correctly real positives. It measures how many selected items are relevant.
  • Recall is the fraction of real positive cases that are correctly predicted positive. It measures how many relevant items are selected.
  • The F1 score balances both precision and recall into a single metric. It ranges from 0 to 1, with 1 being perfect precision and recall, and 0 being the worst.

The F1 score is commonly used instead of raw accuracy when there is an uneven class distribution. It gives a more realistic measure of a model's performance by accounting for false positives and false negatives. Models with high precision but low recall, or vice versa, are penalized compared to models where both are high.

Based on the documents, here are the key F1 scores reported for the cybersecurity binary classification task:

  • ChatGPT-3.5-turbo (16k context): 0.9431
  • ChatGPT-4 (8k context): 0.9410
  • GPT4all: 0.9049
  • Dolly 2.0 (12B parameters): 0.8612
  • Falcon (40B parameters): 0.8511
  • Alpaca-LoRA (65B parameters): 0.8477
  • Stanford Alpaca (30B parameters): 0.6415

The commercial ChatGPT models achieved the highest F1 scores on this task, with GPT4all being the top performing open source model. The scores ranged from 0.6415 to 0.9431 across the different chatbot models examined. The multi-task LSTM model from previous work scored 0.9470.

For the named entity recognition task, the F1 scores were lower overall. ChatGPT-4 achieved 0.41 for organization extraction and 0.54 for product version extraction when tested on a subset of the data. On the full dataset, its F1 score was only 0.10 for extracting all entity types.

So in summary, the F1 scores indicate ChatGPT and GPT4all performed best for classification, while all models struggled more with precise entity extraction, highlighting a limitation in applying chatbots directly for this specialized NLP task.

Authors:

  •  - LASIGE, Faculdade de Ciências, Universidade de Lisboa, Portugal
  • Previous relevant publications:
    • Ferreira et al. (2020) analyzed threat data on Twitter
    • Dionisio et al. (2019, 2020) worked on cyberthreat detection from Twitter using neural networks
    • Alves et al. (2021) presented a system to process tweets for threat awareness

Institutions:

  • LASIGE - Laboratory of Software Engineering, Faculty of Sciences, University of Lisbon

Artifacts:

  • The authors used a dataset of 31281 tweets collected and labeled by Alves et al. (2020). This dataset could potentially be requested for independent verification.
  • Code and models do not seem to be directly shared, but the approaches are described in enough detail to reimplement if needed.
  • Results are presented comprehensively including F1 scores, execution times, prompt examples, and classification outputs. This allows independent assessment.

In summary, the key artifacts are the labeled Twitter dataset and the detailed experimental results. The datasets and implementation details provide potential avenues for reproducing or extending the study if access was granted to the Twitter data.

Some other References:

  1. Daniel Iwugo, "Large Language Models and Cybersecurity – What You Should Know," freecodecamp.org
  2. cybersecurity.dev AI in Cybersecurity: The Role of Large Language Models | cybersecurity.dev

     












Bolstering Cybersecurity: How Large Language Models and Generative AI are Transforming Digital Security | NVIDIA Technical Blog

By Nicola Sessions

Identity-based attacks are on the rise, with phishing remaining the most common and second-most expensive attack vector. Some attackers are using AI to craft more convincing phishing messages and deploying bots to get around automated defenses designed to spot suspicious behavior.

At the same time, a continued increase in enterprise applications introduces challenges for IT teams who must support, secure, and manage these applications, often with no increase in staffing.

The number of connected devices continues to grow, introducing security risks due to an increase in the attack surface. This is compounded by potential vulnerabilities associated with each device.

While there are many security tools and applications available to help enterprises defend against attacks, integrating and managing a large number of tools introduces more cost, complexity, and risk.

​​Cybersecurity is among the top three challenges for CEOs, second to environmental sustainability and just ahead of tech modernization. Generative AI can be transformational for cybersecurity. It can help security analysts find the information they need to do their jobs faster, generate synthetic data to train AI models to identify risks accurately, and run what-if scenarios to better prepare for potential threats. 

Using AI to keep pace with an expanding threat landscape

Cybersecurity is a data problem, and the vast amount of data available is too large for manual screening and threat detection. This means human analysts can no longer effectively defend against the most sophisticated attacks because the speed and complexity of attacks and defenses exceed human capacity. With AI, organizations can achieve 100 percent visibility of their data and quickly discover anomalies, enabling them to detect threats faster.

Although the exponentially increasing quantity of data poses a challenge for threat detection, AI-based approaches to cyber defense require access to training data. In some cases, this isn’t readily available, because organizations don’t typically share sensitive data. With generative AI, synthetic data can help ‌address the data gap and improve cybersecurity AI defense.

One of the most effective ways of synthesizing and contextualizing data is through natural language. The advancements of large language models (LLMs) are expanding threat detection and data generation techniques that improve cybersecurity. 

This post explores three use cases showing how generative AI and LLMs improve cybersecurity and provides three examples of how AI foundation models for cybersecurity can be applied.

Copilots boost the efficiency and capabilities of security teams

Staffing shortages for cybersecurity professionals persist. Security copilots with retrieval-augmented generation (RAG) enable organizations to tap into existing knowledge bases and extend the capabilities of human analysts, making them more efficient and effective.  

Copilots learn from the behaviors of security analysts, adapt to their needs, and provide relevant insights that guide them in their daily work, all in a natural interface. Organizations are quickly discovering the value of RAG chatbots. 

By 2025, two-thirds of businesses will leverage a combination of generative AI and RAG to power domain-specific, self-service knowledge discovery, improving decision efficacy by 50%1.

In addition to not having enough cybersecurity personnel, organizations are challenged in training new and existing employees. With copilots, cybersecurity professionals can get near real-time responses and guidance on complex deployment scenarios without the need for additional training or research.

While security copilots can bring transformational benefits to an organization, they’re only useful when they can provide fast, accurate, and up-to-date information. The NVIDIA AI Chatbot with Retrieval-Augmented Generation workflow provides a great starting point. It demonstrates how to build agents and chatbots that can retrieve the most up-to-date information in real-time and provide accurate responses in natural language.  

Generative AI can dramatically improve common vulnerability defense

Patching software security issues are becoming increasingly challenging as the number of reported security flaws in the common vulnerabilities and exposures (CVEs) database hit a record high in 2022. With over 200,000 cumulative vulnerabilities reported as of the third quarter of 2023, it’s clear that a traditional approach to scanning and patching has become unmanageable. 

Organizations that deploy risk-based analysis experience less costly breaches compared to those that rely solely on CVE scoring to prioritize vulnerabilities. Using generative AI, it’s possible to improve vulnerability defense while decreasing the load on security teams.

Using the NVIDIA Morpheus LLM engine integration, NVIDIA built a pipeline to address CVE risk analysis with RAG. Security analysts can determine whether a software container includes vulnerable and exploitable components using LLMs and RAG. 

This method enabled analysts to investigate individual CVEs 4X faster, on average, and identify vulnerabilities with high accuracy so patches could be prioritized and addressed accordingly. 

A diagram of NVIDIA Morpheus LLM engine for CVE Exploitability using retrieval augmented generation.
Figure 1. CVE exploitability using Morpheus LLM engine supporting model-generated RAG tasks and multiple loops

Foundation models for cybersecurity

While pretrained models are useful for many applications, there are times when it’s beneficial to train a custom model from scratch. This is helpful when there’s a specific domain with a unique vocabulary or the content has properties that do not conform to traditional language paradigms and structures. 

In cybersecurity, this is observed with certain types of raw logs. Think about a book and how words form sentences, sentences form paragraphs, and paragraphs form chapters. There’s an inherent structure that is part of the language model. Contrast that to data contained in a format like JSON-lines or CEF. Proximity of the data keys and values doesn’t have the same meaning. 

Using custom foundation models presents multiple opportunities.

  • Addressing the data gap: while making better use of the influx of data can lead to improved cybersecurity, the quality of the data matters. When there is a lack of available training data, the accuracy of detecting threats is compromised. Generative AI can help ‌address the data gap with synthetic data generation, or by using large models to generate data to train smaller models.
  • Performing “what if” scenarios: novel threats are challenging to defend against without data sets to build the defenses. Generative AI can be used for attack simulations and to perform “what if” scenarios—to test against attack patterns that haven’t yet been experienced. This dynamic model training, based on evolving threats and changing patterns in data can help to improve overall security.
  • Feed downstream anomaly detectors: use large models to generate data that train downstream, lightweight models used for threat detection, which can reduce infrastructure costs while keeping the same level of accuracy.

NVIDIA performed many experiments and trained several cybersecurity-specific foundation models, including one based on GPT-2 style models referenced as CyberGPT. One of those is a model that is trained on identity data (including application logs like Azure AD). With this model, one can generate highly realistic synthetic data that addresses a data gap and can perform “what if” scenarios. 

Figure 2 shows the Rogue2 F1 scores for CyberGPT models of various sizes, with each instance achieving around 80% accuracy. This means that 8 out of 10 logs generated are virtually indistinguishable from logs generated by real network users.

A bar chart showing 80% accuracy for Rogue2 F1 scores of CyberGPT models generated compared with authentic logs.
Figure 2. Accuracy and realism scores of logs generated by CyberGPT models

As for training times, a supercomputer isn’t necessary to realize quality results. In testing, training times were as low as 12 GPU hours for a GPT-2-small model with character-level tokenization. This model is trained on 2.3M rows of over 100 user logs with 1,000 iterations. This model was trained on multiple types of data, including Azure, SharePoint, Confluence, and Jira.

Experiments were also run with tokenizers–primarily character-level tokenizers, off-the-shelf byte pair encoding (BPE) tokenizers, and custom-trained tokenizers. While there are benefits and drawbacks to each, the best performance comes as a result of training custom tokenizers. This not only enables more efficient use of resources due to the custom vocabulary, but it results in reduced tokenization errors and can handle log-specific syntax.

While these results reflect experiments with language models, the same tests with LLMs achieve similar results.

Synthetic data generation provides 100% detection of spear phishing e-mails

Spear phishing e-mails are highly targeted, and therefore, very convincing. The only real difference between a spear phishing (and, in general, any effective phishing campaign) and a benign e-mail is the intent of the sender. This makes spear phishing challenging to defend against with AI because there is a lack of available training data. 

To explore the potential of synthetic data generation in enhancing spear phishing e-mail detection, a pipeline was constructed using NVIDIA Morpheus.

With off-the-shelf models, the spear phishing detection pipeline missed 16% (about 600) of malicious e-mails. The uncaught malicious e-mails were then used to create a new synthetic dataset. A new intent model was learned from the synthetically generated e-mails, and integrated into our spear phishing detection pipeline. The addition of this new intent model feature in the detection pipeline resulted in 100% detection of spear phishing e-mails trained solely on synthetic e-mails. 

The NVIDIA spear phishing detection AI workflow provides an example of how to build this solution using NVIDIA Morpheus.

A diagram showing NVIDIA Morpheus spear-phishing detection AI pipeline using generative AI.
Figure 3. Spear phishing detection pipeline built using synthetically generated spear phishing e-mails that correspond to specific behavioral intents 

A comprehensive approach to enterprise security

The NVIDIA AI platform is uniquely positioned to help address these challenges–building in security at multiple levels. At the hardware infrastructure level, and beyond the data center perimeter to the edge of every server, while also providing tools that help to secure your data with AI. 

Learn more

Watch the session from Bartley Richardson, head of cybersecurity engineering at NVIDIA, to see demonstrations of the use cases illustrated in this post. Learn about integrating language models and cybersecurity featured at NVIDIA LLM Developer Day.

Check out the November 2023 release of NVIDIA Morpheus to access the new LLM engine integration feature, and get started with accelerated AI for cybersecurity. 

Find out how NVIDIA NeMo provides an easy way to get started with building, customizing, and deploying generative AI models. 

NVIDIA Morpheus and NeMo are included with NVIDIA AI Enterprise, the enterprise-grade software that powers the NVIDIA AI platform.

  1.  IDC FutureScape: Worldwide Artificial Intelligence and Automation 2024 Predictions, #AP50341323, October 2023 ↩︎

Novel AI System Achieves 90% Accuracy in Detecting Drone Jamming Attacks

Loss convergence analysis using test data under LoS and NLoS conditions     Novel AI System Achieves 90% Accuracy in Detecting Drone Jamming...