Conclusion

Let’s get straight to the point: “AI needs expensive GPUs” is now outdated.

A 175 billion parameter AI model, comparable to GPT-3, has been successfully run on a single laptop equipped with an RTX 4060.

With 32GB of RAM and 8GB of VRAM, this machine can be purchased for around 150,000 yen at retail stores. Previously, running models of this scale required a GPU cluster with multiple NVIDIA A100 or H100 cards, costing millions of yen just for the hardware. Renting in the cloud would still set you back hundreds of thousands to millions of yen per month.

Now, it’s all possible on a single laptop.

“So, what changes?”—this is the crux of the matter.

What Happened: Results from the DeepSeek 175B Experiment

The recent buzz surrounds a study that quantified the DeepSeek 175B model (a technique to reduce model precision for lightweight operation) and executed it on a standard consumer laptop.

Let’s summarize the specific results:

• Hardware: RTX 4060 laptop (32GB RAM / 8GB VRAM)
• Task: 200,000-scale protein-ligand virtual screening (a computational process used in the early stages of drug discovery)
• Processing Time: Completed within 72 hours
• Throughput: 100 times faster compared to conventional methods
• Accuracy: Maintained the chemical precision required for preclinical drug discovery

The key point is not just that it “worked,” but that it “completed practical tasks with practical accuracy in a practical timeframe.”

Of course, the inference speed is slower compared to GPU clusters. It may not be suitable for real-time response applications like chatbots at this stage. However, for batch processing—like running overnight and checking results in the morning—this is well within practical limits.

While this discussion is centered around drug discovery, the structure applies to other industries as well.

The Essential Question: What Happens When the ‘Cost Structure of AI’ Changes?

Now, this is where it becomes crucial for small and medium enterprises.

Let’s set aside the technical aspects for a moment and consider the changes in cost structure.

The conventional wisdom around AI implementation costs has been as follows:

Now, with the possibility of running on a “single laptop,” it looks like this:

Initial investment of 100,000 yen, running costs of 1,000 yen per month.

For companies using a cloud API at 50,000 yen per month, they can recoup their investment in just two months. Annually, this translates from 600,000 yen to 120,000 yen, a difference of 480,000 yen. Over five years, that amounts to a 2.4 million yen difference.

Moreover, running it on their own laptops means data does not leave the premises. For small and medium enterprises handling personal information and client data, this implies that “the cost of security measures becomes zero.” There’s no longer a need to worry about the risks of sending customer data to a cloud API.

“So, is the Cloud API Unnecessary?”—Not Quite

It’s important to clarify that cloud APIs are not rendered worthless.

The criteria for differentiation are simple:

• Real-time response needed (chatbots, customer service) → Cloud API is advantageous
• Always want to use the latest model (accuracy-first analysis) → Cloud API is advantageous
• Batch processing is sufficient (daily report summaries, document checks, data classification) → Local execution is advantageous
• Handling confidential data (customer information, financial data) → Local execution is advantageous
• Want to fix costs (variable pricing can be problematic) → Local execution is advantageous

In many small and medium enterprises, the latter three scenarios are actually the most common. Summarizing daily reports, checking invoice contents, classifying inquiry emails—these “mundane but daily tasks” are precisely where local AI excels.

What Technically Changed: Two Breakthroughs

Let’s briefly outline why it suddenly became possible to run on a laptop.

1. Evolution of Quantization Technology (Layer Bit Width Optimization)

A technique to compress AI model parameters from 32-bit floating-point to 4-bit or 2-bit. Simply compressing leads to a drop in accuracy, but the latest methods automatically assign optimal bit widths for each layer. Important layers maintain high precision while less critical layers are compressed more aggressively. This allows for model sizes to be reduced to 1/4 to 1/8 while minimizing accuracy loss.

2. KV Cache Compression Technology

A technique that reduces the memory (KV cache) required for AI to process long texts by over 70%. This results in a maximum speed increase of 3.33 times. Even with 8GB of VRAM, processing that previously required 24GB is now possible.

With the combination of these two advancements, it has become a reality to “run massive models on small hardware at practical speeds.”

What Should Small and Medium Enterprises Do Now?

For those who might think, “This is interesting, but it doesn’t concern us,” I want to emphasize this point.

This change is more advantageous for small and medium enterprises than for large corporations. There are three reasons:

1. Faster Decision-Making
While large corporations are going through security reviews and approvals, small and medium enterprises can simply “buy one unit and try it out.” No company takes three months to decide on a 100,000 yen investment.

2. Constraints of Not Being Able to Send Data Outside Become an Advantage
Companies that thought, “We can’t use AI because we can’t send data to the cloud” will find the benefits of local AI to be significant. These constraints become the very reason for adoption.

3. Companies with Many ‘Personalized Tasks’ Will See Greater Effects
Local AI can learn and systematize know-how that only veteran employees possess. The inability to send data to the cloud allows for a more secure implementation.

Here are specific actions to take:

1. First, prepare one laptop with RTX 4060 or higher (used for 80,000 to 120,000 yen)
2. Install a local LLM execution environment like Ollama (free, completed in 30 minutes)
3. Choose one “routine task” you do daily and throw it at the AI
4. Verify whether accuracy and speed are practical for one week
5. If it works, expand its use. If not, compare it with a cloud API

The maximum investment is 150,000 yen. It’s an amount that won’t hurt even if it fails.

The True Meaning of the “Post-GPU Era”

Finally, one last point.

The “post-GPU era” does not mean that GPUs will become unnecessary. It signifies the end of an era where nothing could be done without GPUs.

The biggest barrier to AI adoption for small and medium enterprises has not been technology or talent, but “cost.” Initial investments of several million yen and monthly running costs of hundreds of thousands of yen are now reduced to 100,000 yen and 1,000 yen per month.

Once the wall disappears, what remains is simply the question of “whether to act or not.”

Technology won’t wait. In six months, even the content of this article may be considered “outdated.” That’s why it’s meaningful to start the smallest experiment you can today.

With just a 100,000 yen laptop, your company can begin utilizing AI.