OpenBMB/MetaMem

OpenBMB/MetaMem

Description: [ACL '26] This is the code repo for our ACL '26 Findings paper "MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization"

Language: Python

License: Apache-2.0

Stars: 31

Forks: 3

Open issues: 0

Created: 2026-01-12T08:51:38Z

Pushed: 2026-02-18T13:53:28Z

Default branch: main

Fork: no

Archived: no

README:

📖 Introduction

MetaMem addresses the challenge of fragmented memory and degraded reasoning in long-horizon interactions by constructing a self-evolving meta-memory framework. It iteratively distills transferable knowledge utilization experiences through self-reflection and environmental feedback, guiding LLMs to accurately extract critical evidence from scattered memory units. MetaMem demonstrates strong generalization capabilities by significantly enhancing performance in multi-session integration and temporal reasoning tasks across various retrieval-augmented architectures.


⚙️ Setup

1. Create Conda Environment

conda create -n metamem python=3.11 -y
conda activate metamem

2. Install LightMem

git clone https://github.com/zjunlp/LightMem.git
cd LightMem
pip install -e .

3. Pretrained LLM weights

# Qwen3-30B-A3B-Instruct
hf download Qwen/Qwen3-30B-A3B-Instruct-2507

# Llama3.1-70B-Instruct
hf download meta-llama/Llama-3.1-70B-Instruct

# Qwen3-235B-A22B
hf download Qwen/Qwen3-235B-A22B

# LLMLingua2
hf download microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank

# all-MiniLM-L6-v2
hf download sentence-transformers/all-MiniLM-L6-v2

4. Deploy OpenAI Model Serve

# Qwen3-30B-A3B-Instruct
docker run -d --gpus all \
-e CUDA_VISIBLE_DEVICES=0,1 \
-v /parent_dir_to_models:/workspace \
-p 29001:29001 \
--ipc host \
--name sglang_qwen_30b \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path /workspace/Qwen3-30B-A3B-Instruct-2507 \
--served-model-name qwen3-30b \
--host 0.0.0.0 \
--port 29001 \
--tp 2 \
--mem-fraction-static 0.85 \
--trust-remote-code

# Qwen3-235B-A22B
docker run -d --gpus all \
-e CUDA_VISIBLE_DEVICES=2,3,4,5 \
-v /parent_dir_to_models:/workspace \
-p 29002:29002 \
--ipc host \
--name sglang_qwen_235b \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path /workspace/Qwen3-235B-A22B \
--served-model-name qwen3-235b \
--host 0.0.0.0 \
--port 29002 \
--tp 4 \
--mem-fraction-static 0.85 \
--trust-remote-code

🔧 Reproduction Guide

1. Dataset Preprocessing

wget -c https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json -O data/longmemeval_s_cleaned.json

2. Construct Memory

bash scripts/construct_memory.sh

3. Training MetaMem

# process train data
bash scripts/process_train_data.sh

# k-fold split
bash scripts/split_data.sh

# train
bash scripts/train_metamem.sh

4. Evaluate MetaMem

bash scripts/eval_metamem.sh

5. Inference

bash scripts/infer_metamem.sh

📁 Repository Structure

MetaMem/
├── README.md
├── LICENSE
├── figs/ # README figures
├── scripts/ # The scripts used to run the experiments
└── src/
├── construct_memory.py # Construct the factual memory via LightMem
├── eval_metamem.py # Evaluate the trained meta memory
├── infer_metamem.py # Inference the trained meta memory
├── process_train_data.py # Preprocess the dataset
├── split_data.py # Split the dataset for k-fold validation
└── train_metamem.py # Train meta memory

📄 Acknowledgement

Our work is built on the following codebases, and we are deeply grateful for their contributions.

• LightMem: We utilize LightMem to consturct factual memory.
• SGLang: We utilize SGLang framework to deploy LLM serve.

🥰 Citation

We appreciate your citations if you find our paper relevant and useful to your research!

@article{xin2026metamem,
author = {Xin, Haidong and Li, Xinze and Liu, Zhenghao and Yan, Yukun and Wang, Shuo and Yang, Cheng and Gu, Yu and Yu, Ge and Sun, Maosong},
journal = {ArXiv preprint},
title = {MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization},
url = {https://arxiv.org/abs/2602.11182},
volume = {abs/2602.11182},
year = {2026}
}

📧 Contact

For questions, suggestions, or bug reports, please contact:

xinhaidong@stumail.neu.edu.cn