GPT-4 Citation Found in Kim Il Sung University's Journal

GPT-4 and REALM Show Up in Kim Il Sung University’s Journal

Issue No. 1, Vol. 72 (2026) of the Information Science journal published by Kim Il Sung University carried a paper titled “A Method for Building Training Data for Question Recommendation Models in Intelligent Search Systems.” NK Economy published images from the journal showing a passage explaining GPT-4 and Claude-2, and the reference list included OpenAI’s GPT-4 technical report alongside Google researchers’ REALM paper side by side.

The fact that a North Korean researcher reads foreign AI papers isn’t, by itself, surprising — both documents are publicly available on the internet. What this report let us see was how North Korean search-system research draws on foreign language-model research.

Looking at this case, I found myself thinking about the gap between published research and actual development capability. For context, I completed mandatory service as a sergeant in the Republic of Korea Army and finished my reserve duty as a civil defense member. I don’t say this to downplay North Korea’s military threat. In this piece, I want to focus strictly on the technical content that can be verified from publicly available papers.

The Paper’s Subject and Cited Literature

The paper’s authors are Jinbeom Kim and Seungju Han. They researched a model that, given a few input words from a user, recommends a full question sentence built from those words. Think of it as a feature that helps translate search intent into a proper sentence. The goal resembles the search-term autocomplete you see in everyday life, but that doesn’t mean it works the same way as the methods Google or Naver (a major Korean search engine and portal) use.

The research team proposed a method for constructing training data to generate question sentences using a Transformer model. On the publicly available first page, the paper introduces overseas large language models as background for the study.

The introduction explains that GPT-4 and Claude-2 show strong performance on natural language processing tasks and are being adopted into search systems.

It then mentions the trend of Google, Bing, and Baidu integrating generative AI into search. However, this description and its citations alone don’t support the conclusion that the research team directly used GPT-4 or developed a model of comparable scale.

We also need to be clear about what the references actually confirm.

The GPT-4 technical report was published on arXiv in 2023 (arXiv:2303.08774). REALM is a paper presented at ICML in 2020 (conference source). Neither is a document that requires obtaining some secret, restricted material to read.

The fact that a 2026 journal cites a 2023 paper doesn’t let us calculate that North Korea’s technology lags by 3 years. The moment of citation can differ from the moment a researcher first read the material, and it tells us nothing about whether what was read was actually reproduced. To judge a technology gap, you need performance measured on the same task, along with the same training and operating conditions.

What Public Reports Reveal — and What They Don’t

GPT-4’s report was made public, but that report alone doesn’t disclose all the development information needed to reproduce the same model.

The report explicitly states that it doesn’t reveal the model’s size, detailed architecture, the hardware and compute used for training, or the specifics of the dataset composition and training methods. It cites competitive and safety reasons for this.

Instead, it disclosed evaluation results, limitations, and safety review findings. It also included an explanation that reinforcement learning from human feedback (RLHF)¹ was applied. Researchers who follow can use this material as a reference for which tasks matter and what should be evaluated. But that doesn’t mean they gain access to the data and implementation methods needed to achieve the same results.

REALM is research on training language models while retrieving external documents. It’s one of the early studies that connects to what we now call RAG², but it’s hard to call it the original invention of every method that combines retrieval with language models. Its contribution lies in the fact that it also improved the retriever alongside the model during training.

Publishing research allows other researchers to review the method and continue follow-up work. Companies can also benefit by publicizing their research achievements and attracting talent. At the same time, whatever information gets published can also be read by competitors. That’s why companies each choose differently how far they’ll go in disclosure.

In AI, some cases go beyond papers to also release code and model weights. Weights are the numerical values obtained through training, and releasing them lets other developers run the model or continue training it further. A report that only introduces the direction of research and the release of an executable model offer very different kinds of help for subsequent development.

What It Takes to Read, Build, and Run Models

Even as public research piles up, the cost of training a large model from scratch doesn’t disappear. You still need data collection and cleaning, compute hardware, power, and the people to troubleshoot failures during training.

The same conditions apply when judging North Korea’s development capacity. But since outside information is limited, it’s hard to say for certain exactly what equipment—or how much of it—North Korea actually has.

A 2024 study by 38 North identified various AI research efforts—speech and character recognition, surveillance, military simulation, and more—in North Korean academic journals and public materials. At the same time, it notes that sanctions may constrain equipment acquisition, making real capacity hard to assess. Having published papers doesn’t confirm the ability to run large-scale services.

When assessing a company’s competitiveness, it’s worth examining these factors separately.

Research and implementation know-how: Published papers share research direction, but they don’t convey all the granular experience and implementation details needed to make experiments succeed.

Training infrastructure: Beyond AI accelerators like the H100³, you need inter-device communication, power and cooling, and failure response. Owning tens of thousands of units of a particular chip doesn’t automatically mean you can build any language model—requirements shift depending on the scale of the model you’re targeting.

Data and evaluation: Sourcing and cleaning data for a target domain, and evaluating model outputs, both require cost and expertise. Even if synthetic data is available, you still need to verify its quality.

Service operations: This is the capacity to identify what users want, fix bugs, and deliver a stable service. A larger user base means more chances to gather feedback, but that doesn’t mean every conversation automatically feeds back into training, or that the model gets updated every day.

The Possibility of Using Open Models, and What’s Actually Confirmed

It’s hard to frame whether North Korea can develop a language model as a simple yes-or-no question. A small search-assist model and a world-class general-purpose model require vastly different resources.

Large-scale training requires a stable power supply. Power shortages are often cited as a factor limiting the scale of North Korea’s development capacity. Still, without verifying the power and facilities of any specific research institute, we can’t conclude that no data center operation is possible at all.

Training a model from scratch is different from leveraging an existing one. Inference is the process of generating answers from an already-trained model, while fine-tuning⁴ is additional training that adapts an existing model to a specific task. The equipment needed varies depending on model size, throughput, and training method. The mere fact that consumer-grade GPUs exist doesn’t let us pin down what model size is feasible.

I think a resource-constrained development organization would likely consider using an open model as one option. Weight-open models from China, like Qwen or DeepSeek, are examples of this. But this is my own speculation about a possible development approach. Whether this particular publication actually used an open model is something that needs separate verification.

Qwen2.5’s official announcement states support for 29+ languages, including Korean. However, Korean-language support doesn’t guarantee sufficient performance for North Korean vocabulary or search contexts. Nor can the size of China’s ethnic Korean population alone tell us what data was used for training, or how much.

Using an open model would reduce the burden of training a new model from scratch. Even so, the work of preparing suitable data and evaluating query-recommendation results would remain. Whether costs would drop to a fraction of the original, or whether a given number of GPUs could achieve the goal — these are questions that can only be answered with actual experimental conditions in hand.

What this issue clearly confirms is that the query-recommendation method was researched against the backdrop of overseas language model research. It is neither a declaration that a GPT-4-equivalent model was built, nor evidence that a Chinese open model was adopted.

Oswarld’s Lens

Working on various market-entry strategies, I’ve often seen how holding a technology and building a service customers keep using are two different things. Even when the technology is public and the barrier to entry is low, fitting it to a customer’s actual workflow and running it reliably remains difficult.

The same is true for AI products — model performance alone can’t explain competitiveness. Even with the same open model, the customer experience differs based on what data is connected to it, what tasks it supports, and how errors are handled when they occur. This North Korean academic journal case got me thinking about exactly that gap. That said, a single paper isn’t enough evidence to claim that the technology gap between countries or companies has narrowed by some number of months.

When I evaluate a company like OpenAI, I look not just at model performance but at service reliability, the speed of responding to customer needs, and enterprise support. These operational capabilities are hard to acquire just by reading papers or downloading a model. That doesn’t mean, though, that we should conclude R&D capability has become less important.

Likewise, the fact that a country or company hasn’t developed its own large general-purpose model doesn’t mean the impact of its AI use is necessarily small. Purpose-specific applications — like the surveillance and military-simulation cases analyzed by 38 North — deserve separate scrutiny.

The same logic applies to Korean companies. I think what will determine business opportunity isn’t simply supporting the Korean language, but how well a service understands a specific customer’s workflow and data, and how effectively it resolves problems that come up in actual use.

Closing

There are three distinctions worth keeping in mind when reading this case.

• The Kim Il Sung University journal cited the GPT-4 and REALM papers. The mere fact of citation tells us nothing about implementation method or the size of the technology gap.
• Pre-training a large model, fine-tuning an open model, and running inference on a model all require different resources. Judging what North Korea actually chose to do would require additional material.
• A company’s AI competitiveness also has to be assessed by looking at model performance, data, and development/operations capability together.

For reference, NK Economy — the outlet behind this reporting — is holding a seminar to mark its 8th anniversary, so anyone interested should check it out. My schedule doesn’t allow me to attend, but I’m told North Korea experts will be speaking on the theme “The Age of AI: Where North Korea’s Digital Transformation Stands Today,” covering how North Korea is adopting and using IT technology.

Register for NK Economy’s 8th Anniversary SeminarNK Economy will hold its 8th anniversary seminar on the afternoon of June 17, 2026. Those wishing to attend are asked to register using the form below.forms.gle

References & Further Reading

Primary sources

• Jingyu Kang, “North Korea Chasing Global LLMs: Kim Il-sung University Journal Caught Citing OpenAI, Google Papers”, NK Economy, 2026. : This report lets you directly examine the original images of the Kim Il-sung University journal and how it formatted its citations.
• OpenAI, “GPT-4 Technical Report”, arXiv:2303.08774, 2023. : This is the very report the North Korean journal cited. Worth comparing directly to see what was disclosed and what was withheld.
• Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang, “REALM: Retrieval-Augmented Language Model Pre-Training”, ICML 2020 (arXiv:2002.08909), 2020. : An early study combining external document retrieval with language model training. In the journal image referenced, the title’s “REALM” is misprinted as “EALM.”

Background

• “Sanctions Built North Korea’s AI Playbook: How Pyongyang Boosts Performance Without Nvidia’s H100”, KMJ, 2026. : An analysis of why North Korean AI development has settled into an efficiency-and-lightweight-first approach under computing infrastructure constraints.
• “North Korea’s Risky Bet on Military AI”, The Diplomat, March 2026. : Citing analysis from 38 North, this piece notes that “power output is insufficient to run data centers.” An analysis of North Korea’s AI development and its power constraints.
• Qwen Team, “Qwen2.5 Technical Report”, arXiv:2412.15115, 2024. : A primary source for how a Chinese open-source LLM trains on and evaluates Korean-language data.
• “Emerging Applications and Implications of Artificial Intelligence in North Korea”, Asian Politics & Policy, 2025. : If you’re looking for an academic analysis of North Korean AI policy, this paper is a good place to start.

The author is Oswarld (Kwangseob Ahn). Current roles: Adjunct Professor at Sejong University, Strategy Consultant at INLEVEL9. Career, research, books, and recent work are kept current on the About page. Latest · July 2026: HEMA-2: A Consolidation-Aware Tri-Memory Architecture with Multi-Channel Scheduling for Lifelong Conversational AI.

Footnotes

1. RLHF (Reinforcement Learning from Human Feedback): a technique that aligns AI models through reinforcement learning, using feedback from human evaluators — rating one response as “better” or “worse” than another — as the reward signal. This is the key trick behind ChatGPT answering in the way we’ve come to expect. ↩

2. RAG (Retrieval-Augmented Generation): a method where an AI first searches an external document database for relevant material before generating an answer, using the retrieved results as reference. It helps provide supporting evidence and sources, but there’s still a chance the retrieval or the answer itself is wrong. ↩

3. H100: a data-center-grade AI accelerator made by Nvidia. It’s one type of hardware used to train and run large-scale models. Not every language model requires an H100. ↩

4. Fine-tuning: the process of further training an already pretrained large model on data suited to a specific domain, language, or task, in order to specialize it. This reduces the burden of training a model from scratch, though the extent of the savings depends on the model’s size, the data, and the training method. ↩