101 Pull requests merged by 35 people

• 【Fix Bug】 修复 fa3 支持集中式bug

  #3235 merged Aug 6, 2025

• [Bug fix] Test td cache messager

  #3242 merged Aug 6, 2025

• [FIX 2.1]fix bad_words when sending requests consecutively

  #3199 merged Aug 6, 2025

• [Feature] support seed parameter

  #3161 merged Aug 6, 2025

• support qwen3moe

  #3084 merged Aug 6, 2025

• [BugFix] support real batch_size

  #3217 merged Aug 6, 2025

• add some evil cases

  #3240 merged Aug 6, 2025

• [CI] Add ci case for min token and max token

  #3229 merged Aug 6, 2025

• Fix the confused enable_early_stop when only set early_stop_config

  #3214 merged Aug 6, 2025

• [Bug fix] fix bug for pd step signal

  #3230 merged Aug 6, 2025

• [Bug fix] Fix visit zmq concurrently bug

  #3233 merged Aug 6, 2025

• Perfect approve error message

  #3224 merged Aug 6, 2025

• [New Feature] Support W4Afp8 MoE GroupGemm

  #3171 merged Aug 6, 2025

• [Trace]add trace when fd start

  #3174 merged Aug 5, 2025

• [Feature] optimize expert parallel

  #3196 merged Aug 5, 2025

• Fix approve ci

  #3212 merged Aug 5, 2025

• [Feature] Optimize prefix cache

  #3208 merged Aug 5, 2025

• support qk norm for append attn

  #3145 merged Aug 5, 2025

• [Bug Fix] Fix bug of MLA Attention Backend

  #3176 merged Aug 5, 2025

• revise noaux_tc

  #3164 merged Aug 5, 2025

• Ce add bad cases

  #3215 merged Aug 5, 2025

• [BugFix] support real batch_size

  #3109 merged Aug 5, 2025

• [BugFix]fix test_air_top_p_sampling name

  #3211 merged Aug 5, 2025

• Ce add repitation early stop cases

  #3213 merged Aug 5, 2025

• [Bug fix] Fix lm head bias

  #3185 merged Aug 5, 2025

• [EP] Refactor DeepEP Engine Organization for Mixed Mode & Buffer Management Optimization

  #3182 merged Aug 5, 2025

• [Test] scaled_gemm_f8_i4_f16 skip test while sm != 89

  #3210 merged Aug 5, 2025

• [New Feature] fa3 支持flash mask

  #3184 merged Aug 5, 2025

• fix coverage report

  #3198 merged Aug 5, 2025

• add more cases

  #3207 merged Aug 5, 2025

• [Bug Fix]Fix bug of append attention test case

  #3202 merged Aug 5, 2025

• Fix eplb part3

  #3206 merged Aug 5, 2025

• [Bugfix] Fix uninitialized decoded_token and add corresponding unit test

  #3201 merged Aug 5, 2025

• Add switch to apply fine-grained per token quant fp8

  #3192 merged Aug 5, 2025

• [Bug Fix] Fix bug of MLA Attention Backend

  #3178 merged Aug 5, 2025

• Add more base chat cases

  #3203 merged Aug 5, 2025

• [plugin] Custom model_runner/model support

  #3186 merged Aug 5, 2025

• [FIX]fix bad_words when sending requests consecutively

  #3197 merged Aug 4, 2025

• [Feature] Support ep pd with external module

  #3194 merged Aug 4, 2025

• fix expertwise_scale

  #3181 merged Aug 4, 2025

• [CI] add test_compare_top_logprobs

  #3191 merged Aug 4, 2025

• [Bugfix] Fix uninitialized decoded_token and add corresponding unit t…

  #3195 merged Aug 4, 2025

• [Bug fix] Fix cudagraph when use ep.

  #3130 merged Aug 4, 2025

• remove useless code

  #3166 merged Aug 4, 2025

• [Cherry-pick] FIx bug for scheduler V1

  #3167 merged Aug 4, 2025

• 【Feature】support qwen3 name_mapping

  #3170 merged Aug 4, 2025

• 【Feature】support qwen3 name_mapping

  #3180 merged Aug 4, 2025

• 【Feature】support qwen3 name_mapping

  #3179 merged Aug 4, 2025

• Apply CI fix from Develop

  #3151 merged Aug 4, 2025

• [XPU] Update XPU dockerflie

  #3147 merged Aug 4, 2025

• [Bug Fix] fix the bug in test_sampler

  #3157 merged Aug 4, 2025

• [XPU]Fix out-of-memory issue during single-XPU deployment

  #3131 merged Aug 4, 2025

• Fix approve shell scripts

  #3108 merged Aug 4, 2025

• [cherry-pick]fix load_pre_sharded_checkpoint (#3152)

  #3169 merged Aug 4, 2025

• Update test_base_chat.py

  #3183 merged Aug 4, 2025

• [Bug Fix] fix pd disaggregated kv cache signal

  #3173 merged Aug 4, 2025

• [Bug Fix] fix pd disaggregated kv cache signal

  #3172 merged Aug 4, 2025

• 【Feature】add fd plugins && rm model_classes

  #3123 merged Aug 4, 2025

• fix load_pre_sharded_checkpoint

  #3152 merged Aug 4, 2025

• Update __init__.py

  #3163 merged Aug 4, 2025

• fix_eplb_part2

  #3160 merged Aug 2, 2025

• 【Inference Optimize】Support automatic generation of marlin kernel

  #3149 merged Aug 1, 2025

• [Executor]Fix get_block_shape_and_split_kv_block Kernel typo

  #3153 merged Aug 1, 2025

• [BugFix] fix request_output sampling_params in PD

  #3154 merged Aug 1, 2025

• [Feature] remove dependency on enable_mm and refine multimodal's code

  #3014 merged Aug 1, 2025

• [XPU]Updata XPU dockerfiles

  #3144 merged Aug 1, 2025

• Add CI cases

  #3155 merged Aug 1, 2025

• Fa3 支持集中式

  #3112 merged Aug 1, 2025

• fix ci

  #3141 merged Aug 1, 2025

• add case

  #3150 merged Aug 1, 2025

• [Code Simplification] Refactor Post-processing in VL Model Forward Method

  #2937 merged Aug 1, 2025

• [XPU]Fix out-of-memory issue during single-XPU deployment

  #3133 merged Aug 1, 2025

• fix stop seq unittest

  #3126 merged Aug 1, 2025

• 【Fix】【MTP】Fix mtp bug

  #3140 merged Aug 1, 2025

• [Docs] Optimal Deployment

  #2768 merged Aug 1, 2025

• [OPs] MoE Preprocess OPs Support 160 Experts

  #3121 merged Aug 1, 2025

• [CE] Add base test class for web server testing

  #3120 merged Jul 31, 2025

• disable test_cuda_graph.py

  #3124 merged Jul 31, 2025

• [CI] add xpu ci case

  #3111 merged Jul 31, 2025

• Describe PR diff coverage using JSON file

  #3114 merged Jul 31, 2025

• [Feature] General support for logprobs

  #2974 merged Jul 31, 2025

• [BUG] Fix bug for pd in fd

  #3034 merged Jul 31, 2025

• Fix test_EB_Lite_serving.py

  #3119 merged Jul 31, 2025

• fix is_permuted

  #3098 merged Jul 31, 2025

• [Doc] add chat_template_kwagrs and update params docs

  #3103 merged Jul 31, 2025

• [Feature] block scheduler v1 support prefix caching

  #3061 merged Jul 31, 2025

• [Bug Fix] Fix bug for offline inference in scheduler v1

  #3117 merged Jul 31, 2025

• fix ci

  #3106 merged Jul 31, 2025

• [doc] best practice for eb45 text models

  #3002 merged Jul 31, 2025

• fix w4a8 scale

  #3115 merged Jul 31, 2025

• [Doc] Release fastdeploy-xpu 2.0.3

  #3105 merged Jul 31, 2025

• [Doc]Release fastdeploy-xpu 2.0.3

  #3087 merged Jul 31, 2025

• add approve ci

  #3093 merged Jul 31, 2025

• [Executor] Refactor GetBlockShapeAndSplitKVBlock Kernel

  #2989 merged Jul 30, 2025

• fix is_permuted

  #3100 merged Jul 30, 2025

• [BUG FIX] Fix bug when preempted request rescheduled

  #3080 merged Jul 30, 2025

• [Feature] Support include_stop_str_in_output in completion api

  #3096 merged Jul 30, 2025

• [BugFix]Fix ep size

  #3092 merged Jul 30, 2025

• [Feature] support ep in mixed mode

  #3001 merged Jul 30, 2025

• [doc] add stop_seqs doc

  #3090 merged Jul 30, 2025

• logprob debug use

  #3095 merged Jul 30, 2025

42 Pull requests opened by 30 people

• [Feature] optimize prefix cache

  #3107 opened Jul 31, 2025

• New Loader Support 0.3B

  #3110 opened Jul 31, 2025

• [Docs]fix sampling docs

  #3113 opened Jul 31, 2025

• [Feature] support seed parameter

  #3116 opened Jul 31, 2025

• [Feature] multi source download

  #3125 opened Aug 1, 2025

• [Feature] Support ep pd with external module

  #3128 opened Aug 1, 2025

• [Feature]optimize expert parallel

  #3136 opened Aug 1, 2025

• make append_attn supports mask_offset

  #3138 opened Aug 1, 2025

• 【Fix】【MTP】Fix MTP sample bug

  #3139 opened Aug 1, 2025

• Move create_parameters to __init__ in FuseMOE for CultassBackend and TritonBackend

  #3148 opened Aug 1, 2025

• Fix target_version

  #3159 opened Aug 2, 2025

• [Code Simplification] remove cum_offsets

  #3175 opened Aug 4, 2025

• Update GCU and ILUVATAR CI yaml

  #3177 opened Aug 4, 2025

• [SOT] Use bool index instead of custom op

  #3187 opened Aug 4, 2025

• [XPU]Release fastdeploy-xpu 2.1

  #3188 opened Aug 4, 2025

• [CI] add CI logprobs case

  #3189 opened Aug 4, 2025

• [GCU] Enable gcu CI

  #3190 opened Aug 4, 2025

• Qwen25 branch

  #3193 opened Aug 4, 2025

• [Feature]Pr 3186

  #3200 opened Aug 4, 2025

• [Feature] block sparse attention

  #3209 opened Aug 5, 2025

• [stop_seq] fix out-bound value for stop sequence

  #3216 opened Aug 5, 2025

• test approve

  #3218 opened Aug 5, 2025

• [feat] add metrics for yiyan adapter

  #3219 opened Aug 5, 2025

• [Feature] support disable cache task in decode node

  #3220 opened Aug 5, 2025

• Test CI approve

  #3221 opened Aug 5, 2025

• Ci optimize

  #3222 opened Aug 5, 2025

• [Excutor] Fixed the issue of CUDA graph execution failure caused by different branches during decoding

  #3223 opened Aug 5, 2025

• add base test ci

  #3225 opened Aug 5, 2025

• [WIP] Multimodal-Model support CudaGraph

  #3226 opened Aug 5, 2025

• [fix] setting disable_chat_template while passing prompt_token_ids led to response error

  #3228 opened Aug 5, 2025

• [Bugfix]fix noaux_tc op

  #3231 opened Aug 5, 2025

• Fix iluvatar ci

  #3232 opened Aug 6, 2025

• [Iluvatar GPU] Optimze attention and moe performance

  #3234 opened Aug 6, 2025

• Fix approve ci bug

  #3239 opened Aug 6, 2025

• [MetaxGPU] Support FastDeploy on metax gpu

  #3241 opened Aug 6, 2025

• [bugfix]fix for all_reduce

  #3243 opened Aug 6, 2025

• [Bug fix] fix ep lm head

  #3244 opened Aug 6, 2025

• Wenxin tools 551-completion接口echo回显支持

  #3245 opened Aug 6, 2025

• [BugFix] v1/completions add finish_reason

  #3246 opened Aug 6, 2025

• [fix] fix completion stream api output_tokens not in usage

  #3247 opened Aug 6, 2025

• [fix ]fix qk norm computing error

  #3248 opened Aug 6, 2025

• [Bug fix] support logprob in scheduler v1

  #3249 opened Aug 6, 2025

3 Issues closed by 2 people

• fastdeploy支持modelscope吗？

  #3059 closed Aug 6, 2025

• 修复modify qp to rts失败时存在的内存泄露问题

  #3053 closed Aug 1, 2025

• fastdeploy部署的ernie4.5-0.3b支持openai的json_schema格式输出吗？

  #3102 closed Jul 31, 2025

5 Issues opened by 4 people

• Error not find reason

  #3238 opened Aug 6, 2025

• FastDeploy:2.0.3，发布DeepSeek-R1-Distill-Qwen-32B时，报错

  #3237 opened Aug 6, 2025

• 我想关闭大语言模型的思考模式，如何弄？

  #3158 opened Aug 2, 2025

• [feature reqeust] LogitsProcessor的支持

  #3101 opened Jul 30, 2025

• [Feature Request] FLAGS_max_partion_size和文档需要规范化

  #3097 opened Jul 30, 2025

11 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• 请问有官方答疑群吗，能不能贴一个QR code

  #2979 commented on Aug 5, 2025 • 0 new comments

• [Feature] mm and thinking model support structred output

  #2749 commented on Aug 6, 2025 • 0 new comments

• Fixed the garbled text issues in Qwen3-8B

  #2782 commented on Aug 2, 2025 • 0 new comments

• first commit

  #3029 commented on Aug 5, 2025 • 0 new comments

• w4afp8

  #3044 commented on Aug 6, 2025 • 0 new comments

• Launch expert_service before kv_cache initialization in worker_process

  #3045 commented on Aug 4, 2025 • 0 new comments

• test diff coverage report

  #3063 commented on Jul 31, 2025 • 0 new comments

• [Feature] Models api

  #3073 commented on Aug 6, 2025 • 0 new comments

• Unify server-side and model-side Config(Part-5)

  #3081 commented on Aug 5, 2025 • 0 new comments

• Support for cases where input and output types are different.

  #3082 commented on Aug 6, 2025 • 0 new comments

• Seed

  #3088 commented on Jul 31, 2025 • 0 new comments