High-Performance Computing and Systems Optimization

High-performance computing (HPC) refers to the aggregation of computing power in a way that far outperforms typical desktops or workstations. Its capabilities advance innovation in multiple pursuits, from business to science to technology and beyond. HPC is indispensable in solving complex models and datasets because it completes tasks at greater speed, scale, and efficiency than would be possible without it. In a single second, for example, a laptop or desktop with a 3 GHz processor can perform around 3 billion calculations; HPC solutions can accurately perform quadrillions.¹

The data infrastructure company NetApp explains the importance of HPC this way: “As technologies like the Internet of Things (IoT), artificial intelligence (AI), and 3-D imaging evolve, the size and amount of data that organizations have to work with is growing exponentially. For many purposes, such as streaming a live sporting event, tracking a developing storm, testing new products, or analyzing stock trends, the ability to process data in real time is crucial. To keep a step ahead of the competition, organizations need lightning-fast, highly reliable IT infrastructure to process, store, and analyze massive amounts of data.”¹

HPC performs in tremendously valuable ways, but such advantages don’t come easily. This post will explore some of the functions, challenges, and benefits associated with high-performance computing.

An HPC system requires four key components:

• Compute: This focuses on processing data, executing software or algorithms, and solving problems; a cluster of computers—including their processors, dedicated circuits, and local memory—fall within this scope
• Network: Fast, reliable networking is essential for absorbing external data, moving data between computing resources, and transferring it to and from storage resources
• Storage: External storage is the slowest part of a computer system. Cloud storage, with high volume and high access/retrieval speeds, is vital to the success of high-performance computing
• Memory: This is essential for shared memory scale-up, co-processor interfaces, and dynamic programming methods for polylog scaling

Even for a small company, an HPC system is a big investment. In an industry that pays attention to total cost of ownership (TCO)—a combination of the costs of purchasing and of maintenance—the challenges it presents include:²

• Cost: Even with the savings afforded by cloud infrastructure, the initial expenses and long-term system maintenance costs are significant
• Compliance: Industries involving computerized data storage and use have regulations to follow; if your medical facility, for instance, is implementing an HPC system that will handle personal health records, then compliance with privacy-related standards will be essential
• Security and governance: Cybersecurity needs apply to all facets of a high-performance computing environment. Beyond industry regulations, your organization must adopt and abide by data governance policies to protect the system’s integrity and the data’s safety
• Performance capabilities: In a computing and cloud system this complex, a lot can go wrong—so your company will need strong partnerships with experts who can keep your HPC system functioning at its best

 
What benefits reward all of that commitment?

“We just don't realize how much HPC goes into things that make our world safer, cleaner, and healthier,” wrote Global HPC Business Practice Leader Earl J. Dodd for the technology services provider World Wide Technology. “HPC has been used to run large-scale AI models in fields such as cosmic theory, astrophysics, high energy physics, and data management for unstructured data sets. It is also used in the design of everyday things such as cars, the iPhone, Siri, medicines, and so much more.”

“The diverse applications of HPC,” he continued, “highlight not only its power but also its intrinsic value in creating practical, effective solutions across a broad spectrum of industries. From enhancing product design to advancing scientific research, HPC's contributions are both foundational and transformative.”³

In regard to high-performance computing, one of the key concepts to understand is that of parallel computing. According to IBM, “Parallel computing runs multiple tasks simultaneously on numerous computer servers or processors. HPC uses massively parallel computing, which uses tens of thousands to millions of processors or processor cores.”⁴ Common parallel programming models are shared memory and message passing.

In this architecture system, devices communicate by writing to and reading from a shared memory pool. This enables high bandwidth and point-to-point connections between devices and memory, which eases the board design and layout issues while creating the potential for optimal switch fabric performance. Each interface can run at full bandwidth all the time.⁵

In this parallel computing model, multiple processors communicate and synchronize with each other by passing messages. Each one has its own memory and executes a separate program, but they communicate with each other by sending and receiving messages over a network. Highly scalable, message passing parallelism allows for fault tolerance: If one processor fails, the others’ functions can continue with no interruption.⁶

Writing for the tech community site Built In in 2024, Stephen Gossett offered several examples of parallel processing and their applications. They include:⁷

1. The world of banking relies on graphics processing unit (GPU)-accelerated fintech for everything from credit scoring to risk modeling to fraud detection. In 2011, JPMorgan Chase announced that its transfer from central processing unit (CPU)-only to GPU-CPU hybrid processing had improved risk calculations by 40% and netted 80% savings. Wells Fargo also uses GPUs for processes as varied as accelerating AI models for liquidity risk and virtualizing its desktop infrastructure. 
2. Having developed programs that capitalize on the offloading infrastructure of GPUs, the medical research company Acellera used them to perform simulations on some of the world’s most powerful computers, including one that gave scientists better understanding of how human neurotransmitters communicate. With the technology usable for genetic disease research, Acellera has partnered with Janssen and Pfizer for pharma research.
3. “If you saw the bright ‘80s-themed scenes of Thor: Love and Thunder or the choreographed antics of Bullet Train,” wrote Gossett, “you also saw the handiwork of parallel processing. Both were colored using Blackmagic Design’s DaVinci Resolve Studio, one of a handful of Hollywood-standard post-production suites that incorporates GPU-accelerated tools.” 

Based in the United States and Germany, the company named FS provides information and communication technology products to customers around the world. A 2024 article on its website says, “As the demand for faster and more efficient computational capabilities grows, optimizing HPC infrastructure is crucial to maximizing performance and efficiency.”⁸ It goes on to name seven ways to get the most from a high-performance computing system:

1. Understand workload requirements: To ensure that your system is operating at peak, it’s essential to have a clear picture of what you need it to do. Compute-intensive workloads, for example, require a lot of processing power, while data-intensive workloads demand high I/O throughput and large capacity for storage.
2. Select and configure the appropriate hardware: Get the right equipment for the job. CPUs provide versatility, GPUs supply huge parallel processing power, field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs) optimize performance and energy-efficiency for certain tasks.
3. Ensure network optimization: Get the best performance possible from your system, in regard to speed, reliability, capacity, and security. Place switches and routers strategically to keep latency to a minimum and use the best parallel programming models for the needed tasks.
4. Ensure software optimization: Keep your software footprint as large as, but no larger than, it needs to be. According to Dr. Russ Andersson in writing for the software company RapidFort, “Software is 100% optimized when it contains only those components necessary to run. Currently, the average software container is only ~20% optimized, meaning 80% of the code is not used, and must be maintained.”⁹
5. Manage energy efficiency: Choose solutions that help manage power consumption while maintaining performance. Adjust the system’s power consumption based on workload needs and reduce energy usage when demand is low.
6. Ensure security and scalability: Maintain system integrity by implementing strong security measures, including data encryption, access control, and network security protocols.
7. Continuously monitor performance: Tools that allow real-time monitoring of the system’s status help identify bottlenecks and areas needing improvement. Regularly scheduled performance assessments help ensure that your HPC infrastructure stays tuned for optimal performance.

Once you’ve put in the time and resources to optimize your HPC system, what can it do for you? This post has touched on a few practical examples of high-performance computing. Providing a view with a much wider scope, however, is an excellent article written for IBM in 2024 by Stephanie Susnjara and Ian Smalley:⁴

HPC is key in simulating and optimizing product and process design. It’s used to run computational fluid dynamics (CFD) applications, which analyze and resolve challenges related to fluid flows. This includes optimizing battery performance and safety through battery simulation and simulating aerodynamics to reduce friction and air drag.

HPC powers automated trading and risk analysis methods including fraud detection and applications in Monte Carlo simulation—a type of computational algorithm using repeated random sampling to find the likelihood of a range of results of occurring.¹⁰

HPC can help prevent cyberattacks and other security threats by analyzing large amounts of data to identify patterns.

Closely related to government and defense work, energy-related HPC applications include wind simulation, terrain mapping, reservoir simulation and modeling, seismic data processing, and geospatial analytics.

In addition to energy research and intelligence work, HPC is instrumental in weather forecasting and climate modeling, both of which involve processing enormous amounts of historical meteorological data and millions of daily changes in climate-related data points.

When scientists first tried to sequence the human genome, it took 13 years to finish the job; today, HPC systems can do it in less than a day. HPC visualization helps scientists collect insights from simulations and analyze data quickly. Other healthcare/science-related HPC applications include drug discovery and design, rapid cancer diagnosis, molecular modeling, and medical record management.

Further, as the AI visionary Ray Kurzwell described in his April 2024 TED talk, HPC has been tremendously instrumental in the fight against the COVID-19 virus. “Moderna was trying to create their COVID vaccine. They actually put together a list of different mRNA sequences. Now, what would we do in the past? Someone would come in and say, "OK, there's several billion. Let's try this one." Or maybe they'd pick three. You can't do a clinical test on billions of different possibilities. But that's exactly what they did by simulating the reaction. And that took two days. So in two days, they created the Moderna vaccine. And that is still on the market. It's been the best vaccine. It was done in two days.”¹¹

Thanks to HPC clusters, we have access to the high speed needed to stream live events, render three-dimensional graphics and special effects, and reduce production time and costs. High-performance computing can also help media companies gain data-driven insights they can use to improve content creation and distribution.

Put the power of Chico State’s Online MS in Computer Science program into your professional expertise. Delve into advanced topics that are essential for your career ascent: applied machine learning, software design, programming, AI, database management, and more. In the small-class environment of this flexible online program, you’ll learn from supportive, engaging faculty experts and build the problem-solving skills that will make you a stand-out hiring choice.

Take the first steps today. Review the admissions schedule and financial aid opportunities, and schedule a call with an admissions outreach advisor.