7月30日2026 · 星期四

@trq212 转推了

@OpenAI

GPT-5.6 Sol has been used to solve open problems in mathematics. So why was it struggling with ARC-AGI-3, a benchmark of 2D puzzle games? We investigated. The harness was not letting it remember what it had learned. We found that enabling two API settings tripled our scores with 6x fewer output tokens. --- From twitter --- Terrific work by @ilanbigio and @sandersted on the investigation and post. Seems like it was a lot of fun. Take care of your harnesses out there, your model will thank you. > 引用 @thsottiaux: Turns out GPT-5.6 Sol is actually SoTA on ARC-AGI-3. > > Just took two setting changes. You just have to allow it to reason and work over multiple context windows with the help of our canonical compaction implementation. > > https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/ --- From twitter --- A benchmark score reflects the model as well as the harness and settings used to run it. For long-running agents, retaining reasoning and compacting context lets the model build on what it has already learned. https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/ --- From twitter --- We hope these experiments serve as a reminder that evals rarely measure models in isolation—they also measure a bundle of less visible choices about API settings, harness design, and prompting. If you’re an API developer trying to maximize performance, we recommend using the same settings that we deploy in our own products: - Use our Responses API, not our legacy Chat - Completions API - Retain reasoning - Use compaction If you want to test your own mettle against frontier models, try the public games yourself at http://arcprize.org/tasks

@thsottiaux

Efficiency! In two steps a) Train fantastic model b) Use fantastic model to make everything better, including its own infrastructure, inference stack, kernels, etc, etc https://openai.com/index/gpt-5-6-frontier-intelligence-efficiency/

How GPT-5.6 fuses frontier intelligence with frontier efficiencyopenai.com · 直连原文

@Alibaba_Qwen 转推了

@ArtificialAnlys

Alibaba has released Qwen Audio 3.0 Realtime, with the Plus variant debuting as the new #1 model on the Artificial Analysis Speech to Speech Index at 84.1%, ahead of GPT-Realtime-2.1 High at 79.1% Released earlier this month, Qwen Audio 3.0 Realtime is @Alibaba_Qwen’s flagship native Speech to Speech model, available in two variants: Plus and Flash. Qwen Audio 3.0 Realtime Plus leads on all three component benchmarks comprising the Artificial Analysis Speech to Speech Index, Big Bench Audio for Speech Reasoning, Full Duplex Bench for Conversational Dynamics, and Tau Voice for Agentic Performance. We tested the China-hosted endpoints on Aliyun (Alibaba Cloud). Key takeaways: ➤ Speech to Speech Index: Qwen Audio 3.0 Realtime Plus is the new leader at 84.1%, ahead of GPT-Realtime-2.1 High (79.1%) and GPT-Realtime-2 High (77.2%). The Flash variant comes in at 4th at 76.3%. ➤ Speech to Speech Index by Benchmark: On Big Bench Audio, the Plus variant achieves 99.2%, up ~0.5 percentage points from the previous best of 98.7% (Alibaba Qwen3.5 Omni Plus Realtime), with Flash variant scoring 96.1%. On Tau Voice, the Plus variant currently leads with a score of 54.6%, ahead of Grok Voice Think Fast 1.0 at 52.1%. On Full Duplex Bench, the Plus variant leads our Full Duplex Bench subset at 98.4%, with Flash at 96.9%, both ahead of the best non-Alibaba model, GPT-Realtime-2 (Minimal) at 96.1%. ➤ Speed: The Plus variant records an average Time to First Audio of 4.02 seconds on Big Bench Audio, with Flash at 4.16 seconds, among the slowest models on our leaderboard, and well behind GPT-Realtime-2 (Minimal) at 1.10 seconds and GPT-Realtime-2 (High) at 1.14 seconds ➤ Price: Plus costs $4.42 per hour of input audio on our Big Bench Audio subset, more expensive than GPT-Realtime-2 High ($4.14) and ~2.4x cheaper than GPT-Realtime-2.1 High ($10.75). The average cost for Flash variant is $4.77, higher than the Plus variant despite lower list prices, driven by comparatively more verbose responses. See below for more detail ⬇️

@Kimi_Moonshot 转推了

@arena

Code Arena now measures fullstack capabilities! View overall rankings across AI models on full-stack web development tasks: multi-step reasoning, tool use, and end-to-end app generation. - Kimi K3 (Max) takes #1 - GPT 5.6 Sol (xHigh) at #2 - Claude Fable 5 at #3 See more scores at: http://arena.ai/leaderboard/code/webdev/fullstack

@arena

Code Arena just leveled up with fullstack capabilities 🚀 Introducing the new Fullstack Code Arena. We’re moving beyond frontend prototypes to fullstack development complete with databases, API keys, and fast deployments. Build, iterate, and ship real-world software — all in one place. Models now act as agents in the Code Arena, using structured tool calls to plan, execute, and refine in real time with real world tasks. Read more about it in the thread 🧵

@dotey串推 2 条

@dotey

今年以来，我在使用 Coding Agent 方面有一个很大的变化，就是从 TL（Teach Lead） 的角色变成了 EM（Engineering Manager） 的角色。 这两个角色主要差别在技术参与深度多少。 之前我更像一个 TL，虽然不是说事必躬亲，但是系统设计、代码审查什么的肯定是少不了的，说到底还是对 AI 写的代码不放心。 这样虽然质量更有保障，但是人会成为 Agent 的瓶颈，很多事情需要人去决策，细节需要人去掌握。 转折点在 Fable 5 前后，我发现 AI 写的代码质量已经相当可以了，只要稍加验证就不会有太大偏离，所以我越来越少的去干预 AI 写代码，而是会更站在全局去看一个项目： 决定项目怎么做，去验收好结果。 这极大的释放了 Agent 的生产力，大部分时候我想好要做什么功能，先和 Agent 一起做一个技术方案，然后确认方案没问题后，用 /goal 加上方案，让 Agent 去执行，写代码和自动化测试，等做好再去验收下功能，代码不怎么细看。 有 Bug 了就是把 Bug 描述给 Agent 让它自己去重现解决，并且让 Agent 补上相关测试覆盖，修复了后人再去验证一下。 这样还有一个好处，就是在技术选型时，不会局限于你自己的喜好和擅长。 当你是 TL 的角色是，还是会有点过度关注技术实现，包括技术选型会偏向你自己熟悉的喜欢的，而不一定是最适合的。 当你是 EM 的角色做技术选型，就不再关注自己擅长什么，而是什么技术最适合项目。 我因为前端熟悉，所以最开始开发字幕翻译 App 时，就优先考虑 Electron 这样的技术栈，因为自己熟悉，有问题能解决也能写的出来。 后来发现 Electron 性能很难满意，就换成了 Swift + AppKit 原生技术栈，本来我 Swift 是不熟悉的，但有 AI 辅助，整个过程毫无压力。 现在在设计 BaoCut 下一个大版本的时候，要考虑跨平台方案，首选是 Rust，哪怕我从来没写过一行 Rust 代码，但我知道这是一个很好的跨平台选择。 目前基于 Rust 的第一个版本已经写完了，整个过程几乎没有任何语言上的障碍。 通过这样的模式我在开发 BaoCut 的时候，基本上可以每天一个小版本迭代。https://baocut.app/releases/ 这两天速度慢下来了，是因为需要构思新的大版本，这时候人就又成了瓶颈了：如果人没想清楚该做什么，Agent 再厉害也帮不上。

@dotey

通常北美的工程技术相关的职业分成以下五个类别： 开发工程师 SE / SDE(Software Engineer / Software Development Engineer) 工程经理 EM / SDM(Engineering Manager / Software Development Manager) 技术主管 TL / TLM (Tech Lead / Tech Lead Manager) 技术项目经理 TPM (Technical Program Manager) 产品经理 PM (Product Manager) https://baoyu.io/blog/engineering-management/engineering-roles

@OpenAI串推 2 条

@GitHub_Daily

上个月，那个把文本、图像、视频全模态模型 API 无限期免费开放的 Agnes AI。 一度爆火，至今都还有很多人在白嫖。 今天在 Artificial Analysis 榜单上，看到了它家有个新模型 Agnes 2.5 Pro Alpha 在打榜。 了解了下这次 2.5 Pro Alpha 模型，在复杂推理、专业开发和 Agent 任务等方面，都有不少提升。 同时也支持到 1M 超长上下文窗口，对于开发大型项目、处理长文档等场景来说超有用。 另外我发现它的基础模型已迭代到 Agnes-2.5-Flash，依然还是不限期免费。 开发者可以通过API直接调用，文本之外，Agnes的图像和视频模型也依然免费开放。 拿Agnes-2.5-Flash处理一些日常开发任务已经够用了，感兴趣的朋友可以通过新官网查看模型和调用方式。 官网：https://agnes-ai.com

@GitHub_Daily

OpenAI 悄悄开源一个专门查代码安全漏洞的命令行工具：Codex Security。 另外配了 TypeScript SDK，能直接扫描整个项目仓库，也能只审查某次的改动。 找出来的问题会持久长期记着，哪些修了、哪些还挂着，一目了然。 GitHub：http://github.com/openai/codex-security 除了找出漏洞，还会验证问题是不是真的存在，并给出对应的修复。 接到自动构建流程里，每次提交跑一遍安全检查，不用等上线前才临时补。 支持 ChatGPT 账号登录，也可以用 API Key，方便放在自动化环境里跑。

@thsottiaux

Hello people of Sol! I've reset usage limits for all ChatGPT Work and Codex users. Together with that, a quick update on GPT-5.6 Sol usage limits. Over the past few weeks, many of you have told us that Sol was using your Codex limits faster than expected. To be clear, we have not reduced usage on any subscription plans. We’ve been digging into what was happening and have landed several improvements. As a result, we expect your usage to last around 18% longer during typical use of Sol. Some of you should already see significantly larger improvements from today. Tomorrow, we’ll also restore the five-hour limit that we temporarily paused while investigating. Here’s what we found: - GPT-5.6 Sol is much more willing to work for longer, make additional tool calls, and coordinate complex workflows across tools and subagents. That makes it better at solving hard problems, but some tasks were using far more than we intended. - Sol also works harder at the same reasoning effort than previous models. High on Sol can use more tokens than High did on GPT-5.5. - Programmatic tool calling, also referred to as code mode, gives Sol much more flexibility to run tool calls in parallel or continue working while waiting. But it also led to more responses per turn, more cached input tokens, and higher usage than expected. - This was particularly noticeable when Sol was waiting for tool calls to finish or running many web searches. We’ve improved how we handle both cases and are continuing to make code mode more efficient. - The impact was also very uneven. The median user actually found Sol quite token efficient, while some power users working on harder tasks saw their usage drain much faster. We were very focused on average and median usage before launch and missed some cases where the long tail could use significantly more usage. Sol is a significant step forward in what Codex can do, but capability and efficiency do not always improve at the same pace, and some issues only become clear once people are using the model at real-world scale. We should have recognized this sooner and been more upfront about it. You keep pushing the frontier and we’ll keep improving efficiency and sharing updates as we go.