Conclusion First: The Era of ‘AI in the Cloud’ is Beginning to End

NVIDIA has announced the DGX Spark. It offers desktop-sized performance of 1 PetaFLOPS (FP4), 128GB of memory, and a price starting at around $3,000 (approximately ¥450,000).

Do you understand what these numbers mean?

Three years ago, acquiring 1 PetaFLOPS of computational power required a server room and an investment of tens of millions of yen. Now, it can fit on your desk and costs less than a used compact car.

Many small and medium-sized business owners still believe that “AI should be accessed through cloud APIs.” However, with hardware now available at this price point, it’s time to question that assumption.

Do You Know How Much You’re Paying for Cloud Services Each Month?

First, let’s look at the real numbers.

Using OpenAI’s API (GPT-4 class) for business can lead to token consumption that exceeds expectations. Summarizing internal documents, automating customer responses, generating reports—doing these regularly can easily cost between ¥50,000 and ¥200,000 per month. During peak periods or when used in an agent-like manner, some companies may spend over ¥500,000 monthly.

Moreover, the troublesome aspect of cloud APIs is that the more you use them, the higher the costs become. As AI becomes more integrated into business operations, bills grow larger. The more successful you are, the higher the costs rise. This structure poses significant risks for small and medium-sized enterprises.

What Can You Do with DGX Spark?—A Honest Look at the Specs

Let’s break down what the DGX Spark offers:

• Chip: GB10 Grace Blackwell Superchip
• Performance: 1 PetaFLOPS (FP4)
• Unified Memory: 128GB (shared between CPU/GPU)
• Price: Starting at around $3,000 (approximately ¥450,000)
• Size: Slightly larger than a Mac Mini desktop chassis
• OS: NVIDIA DGX OS based on Linux (Ubuntu)

The 128GB of unified memory is substantial. With this, you can run open models with around 20 billion parameters (like Llama 3 or Mistral) without quantization. Even models with 70 billion parameters can run with quantization.

In other words, while it may not reach the capabilities of GPT-4, it can run inference at a level between GPT-3.5 and 4 locally and infinitely.

This is a crucial point. Many business applications—classifying internal documents, generating standard emails, summarizing meeting minutes, automating FAQs—do not require the highest performance models. Having “a reasonably intelligent model that can be used infinitely at zero cost” is far more valuable in practice.

Let’s Crunch the Numbers for Break-Even Analysis

Let’s do some specific calculations.

In the case of using a Cloud API:

• Monthly Cost: ¥100,000 (assuming average business use for SMEs)
• Annual Cost: ¥1,200,000
• Cost over 3 Years: ¥3,600,000

In the case of implementing DGX Spark:

• Initial Investment: Approximately ¥500,000 (hardware + setup costs)
• Electricity Cost: About ¥2,000 to ¥3,000 per month (the official power consumption has not been announced, but it is estimated to be around 200W as it is desktop-level)
• Annual Running Cost: Approximately ¥30,000
• Total Cost over 3 Years: Approximately ¥590,000

Difference: Approximately ¥3,000,000 over 3 years.

This calculation holds true for companies paying over ¥100,000 monthly for cloud APIs, but even if you take a more conservative view, spending ¥50,000 on cloud services still results in ¥600,000 in a year. The initial investment of ¥500,000 for DGX Spark can be recouped in just 10 months.

Conversely, if you’re only using APIs for less than ¥30,000 a month, then the cloud may still be more reasonable. I must be honest about this.

The break-even point is around ¥40,000 to ¥50,000 per month. If you exceed this, it’s worth seriously considering the introduction of local AI.

‘No Need to Send Data Outside’—This is More Significant for SMEs Than You Might Think

While I’ve focused on costs, there’s another critical point that cannot be overlooked: privacy.

When using cloud APIs, customer data, sales data, and internal know-how are all sent to external servers. Even if OpenAI and Google say they “won’t use it for training,” the anxiety felt by SME owners doesn’t disappear. Can you clearly answer when a business partner asks, “Where are you sending our data?”

With DGX Spark, all data processing is completed on your own desktop. It can operate without being connected to the network. This offers value that cannot be obtained in the cloud, especially for SMEs in sectors with high data confidentiality such as healthcare, law, finance, and manufacturing blueprint management.

For companies that have delayed AI adoption due to privacy concerns, local AI effectively removes that blocker.

However, It’s Not All Sweet—Honest Risks Must Be Addressed

Let me pour some cold water on this. Buying a DGX Spark does not solve everything.

1. Operational Expertise Required
Setting up open models, fine-tuning, and prompt design—these are still not at a level where you can simply “buy it and turn it on”. NVIDIA offers software stacks like NIM microservices, but you still need at least one person in-house with a minimum level of technical understanding.

2. Model Performance Does Not Match Top Cloud Offerings
For applications that require the highest performance, such as complex reasoning, advanced code generation, and multimodal analysis, cloud APIs still have the edge. It’s best to think of local open models as covering “80% of use cases with 90% quality”.

3. Keeping Up with Future Model Evolutions
In the cloud, models are automatically updated on the other side of the API. With local setups, you need to switch to new models yourself. However, this also means there’s stability in that “specifications won’t change without your consent”.

The True Meaning for SMEs—’AI Costs Become Fixed Costs’

After reading this, some business owners may think, “This doesn’t apply to us.” However, the essence of this trend is not the DGX Spark product itself.

It’s the structural change where ‘AI computational costs shift from variable costs to fixed costs.’

Cloud APIs operate on a pay-as-you-go basis. The more you use, the more it costs. This may not be an issue for large corporations, but it can be fatal for SMEs. Budgets become unpredictable. If you succeed and scale, costs scale alongside.

With local AI, as long as you make the initial investment, you can use it infinitely with just electricity costs. Whether you use it 100 times a month or 10,000 times, the cost remains the same. The cost does not correlate with the amount of AI usage. This is the reversal structure for SMEs.

Large corporations can budget for cloud pay-as-you-go models. However, SMEs hesitate to adopt AI when they cannot predict monthly payments. Local AI, which can be turned into fixed costs, is precisely the option for SMEs.

So, What Should You Do?

I propose three steps.

Step 1: Accurately Assess Your Current Cloud AI Costs
Identify the monthly costs associated with AI from OpenAI, Google, AWS, etc. If you exceed ¥50,000 a month, move to the next step.

Step 2: Separate ‘Use Cases That Must Be in the Cloud’ from ‘Use Cases That Are Sufficiently Handled Locally’
Keep high-performance applications in the cloud. Move routine processes and those with high data confidentiality to local. A hybrid approach is the practical solution.

Step 3: Start Small and Experiment
The official release of DGX Spark is expected after May 2025. While waiting, try existing local LLMs (like Ollama) on your current PC. Just getting a sense of “how much can be done locally” will enhance your decision-making accuracy.

In the world of AI, the common knowledge from six months ago may no longer apply. The judgment that “the cloud is sufficient” or that “local is difficult” must be regularly updated, or you may find yourself outpaced by competitors.

A ¥450,000 desktop could eliminate ¥3,000,000 in cloud costs. Many companies that can make this calculation may be more numerous than you think.