Research At Reka Reasoning

Research at Reka: Reasoning

← Back to Blog

Sep 1, 2025

Research at Reka: Reasoning

Research at Reka: Reasoning

This short post is the first in a series of posts that will discuss research topics that are important to us. What are fundamental building blocks of intelligence? At Reka, we train large multimodal models but we also spend a lot of time thinking about what the missing components are at the foundational level and how they translate to our product offerings. In this post, we share our perspective on reasoning. What is Reasoning Reasoning became a popular topic with releases of models such OpenAI o1 and DeepSeek R1. As a result, a lot of people associate it with models that generate thinking tokens (i.e., longer chain of thoughts) before providing answers. Leaderboards and benchmarks separate results from reasoning models and non-reasoning models based on this behavior. We do not think this is the right characterization. Rather than distinguishing by how the model produces outputs, we think reasoning is an underlying capability that allows the model to solve complex goals by breaking them down into steps and dynamically performing self-verification to backtrack along the way. We believe such a capability is necessary to allow our models to not just replicate human intelligence, but to discover new things that surpass our capabilities. How to Get Models to Reason There are two complementary approaches to achieve this goal. Test Time Compute The first method to get a reasoning behavior is by orchestrating inference using test-time compute algorithms. For example, search techniques such as MCTS, parallel thinking, using process reward models to guide decoding, or simply workflow orchestration by creating an execution graph and having another model to verify the next node to explore fall under these categories. In this scenario, while the underlying model is not explicitly trained to perform multi-step inference, we enforce the step-by-step and self-verification behavior by supervising it at test time. The last approach is perhaps the most commonly used method to create an agent that can perform complex tasks right now. An agentic system will call an LLM multiple times with specific instructions to decide on what to do next (e.g., proceed to the next step, retry the current step). While this approach is sufficient for a simple workflow that only needs shallow reasoning, it is not as elegant or generalizable since every workflow is manually designed with human supervision. None of our agentic offerings follow this template-based approach. We instead choose to build the reasoning capabilities directly to the model (see below). Training a Reasoning Model A second variant is by training a model to natively exhibit this behavior. For models that are trained to do this step, they can either explicitly output their thinking process (e.g., … ) or think implicitly in their latent space. What is the best way to train such a model? The main question is whether we can get this capability by scaling up unsupervised pretraining and preference-based reinforcement learning or we need a different approach. Distillation One method that is surprisingly very effective is by learning from reasoning examples. These examples can be created by either manually creating them by asking humans to mimic the desired thinking process, or more commonly by generating synthetic data from other models that have exhibited this behavior (i.e., distillation). It is perhaps surprising how effective distillation is, not just for reasoning but for transferring other kinds of capabilities. Distillation can be done by either training on the output directly (more common since it is easier to do in practice) or by training to mimic the logits of a teacher model. We found that with a strong base model it does not take a lot of examples to trigger this behavior, and it improves performance on a lot of tasks across the board. We also found that stronger coding models are better reasoners, which is one of the reasons that we have been focusing on pretraining and posttraining Reka Flash to become a better coding model. Reinforcement Learning The second approach is using reinforcement learning. Among other learning methods we have today, RL is uniquely suitable to tackle this problem since it only needs to rely on high-level feedback (reward) to discover complex sequential strategies. In RL, an agent interacts with an environment by taking actions to achieve its goal and receives a reward. The agent can iteratively learn through trial and error, either via sparse or dense rewards. This well-defined feedback loop allows the agent to quickly converge on effective problem-solving approaches. Improving reasoning via reinforcement learning requires creating hard tasks that translate , defining rewards for those tasks, and setting up an infrastructure to optimize the system end to end. Early work in the open-source community has been focusing on math reasoning models because it is easy to verify rewards for that task (correct answer or not, which can be done directly with string matching after some regular expression formatting, and does not require running an extra tool) and it does not involve any kind of interaction with external tools. We found performing reinforcement learning in this setup to be straightforward. The main challenge is coming up with harder and harder tasks that fit this setup and designing verifiable rewards for those tasks. However, we also believe that the gain that can be obtained in a pure model-only setup without external tools is limited. As a result, our focus has been on establishing our general end-to-end reinforcement learning infrastructure with tool use. This setup is harder to do since leaving the training environment to use assigned tools can introduce significant latency. The most simple example of this setup is in training a coding model, where the model requires access to a code interpreter to execute the output code and get a reward. This setup can be extended to many diverse tools, from tools to browse the web to MCP calls to accomplish other tasks. Reinforcement learning also has another advantage as the most promising learning approach that allows the model to perform explorations in the training process. Regardless of whether compressing world knowledge (i.e., the web) to a neural network model via unsupervised pretraining can lead to AI agents that can generalize to complex tasks...