{ "rules": "RULES.md section 6: a reviewer from a different model family reads the raw files and confirms or lists disagreements", "rounds": [ { "round": 1, "date": "2026-11-06", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/, the comparison pages", "instructions_sha256": "8359e38a545c46b7369975b2154e215d99d12da2fddc3890ab5c3fda0828cb49", "conclusion": "disagree", "summary": "I independently recalculated all 30 rows; the ten requested rows appear above. Units are GMAC/s/mm², pJ/useful MAC and bits/useful MAC. Only OpenGeMM's truncated cycle average caused arithmetic discrepancies. Correcting it preserves all M1–M3 wins. Subject selection, comparator membership, best-row scoring, strict 1% thresholds and M4b comparisons match the amended rules. Subject logs support G1–G3 and sampled M4a; M4b is 14.131% versus 22.254% and 25.655%. However, G4's complete clean-machine table reproduction is not established, so reached does not stand under the written rules. Other findings concern unequal replay workloads, post-result selection disclosures and page wording. No files were changed; verdict.py was inspected rather than executed because it writes files. Different-model-family reviewer provenance was not independently verifiable.", "recomputed_rows": [ { "row": "nangate45-cv_gemv_r7c_p8", "m1": 459.32950735021643, "m2": 0.8334500847424959, "m3": 5.375, "matches_summary": true }, { "row": "asap7-cv_gemv_r7r_p8", "m1": 21849.06444823796, "m2": 0.17001999959271652, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-vta8_official", "m1": 447.005717316261, "m2": 0.9562967602379937, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8_official", "m1": 20193.72594919453, "m2": 0.1918849503884452, "m3": 6, "matches_summary": true }, { "row": "nangate45-vta8", "m1": 443.37750660349536, "m2": 0.9864192272123983, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8", "m1": 17243.669317763117, "m2": 0.196943280027989, "m3": 6, "matches_summary": true }, { "row": "nangate45-opengemm8", "m1": 313.365889452936, "m2": 1.1491631235681978, "m3": 6, "matches_summary": false }, { "row": "asap7-opengemm8", "m1": 7544.929306601196, "m2": 0.23964751371225915, "m3": 6, "matches_summary": false }, { "row": "nangate45-w8a8_p8", "m1": 316.311264395914, "m2": 1.2649527814179615, "m3": 9, "matches_summary": true }, { "row": "asap7-w8a8_p8", "m1": 16489.799316150493, "m2": 0.23235266326445386, "m3": 9, "matches_summary": true } ], "findings": [ { "file": "corvel-comparison/studies/compare/verdict.py", "location": "lines 72–86, G4; evidence/compare/rerun-summary.json; evidence/compare/second-machine.json", "problem": "G4 accepts matching subject rows without requiring all comparator rows or evidence of a clean-machine run. The supplied rerun has only eight rows: it omits official VTA, OpenGeMM and Gemmini in both processes. The separate-machine record covers only Nangate45 CV-GEMV and official VTA, and only layout hashes and selected flow metrics—not the complete workload-power table. Thus RULES.md line 61's complete-table reproduction requirement is not established, despite verdict.json and the page marking G4 pass.", "effect": "the verdict", "how_to_check": "Compare required subject/comparator keys with rerun-summary.json rows and second-machine.json runs. Require complete rerun coverage, including power/replay metrics, with clean-run provenance. Under RULES.md section 6, an unfulfilled G4 prevents reached.", "status": "re-measured", "resolution": "verdict.py now requires every verdict row, both processes, reproduced with every number equal on a second machine (and in any clean rerun). The six missing comparator rows are being rerun from clean directories, and the full table on a second machine. Done: both second runs reproduce all 14 rows with every number equal.", "status_kr": "다시 잼", "resolution_kr": "verdict.py가 이제 판정에 쓰는 모든 줄(두 공정)을 다른 기계에서 모든 숫자가 같게 재현할 것을 요구합니다. 빠진 비교 상대 6줄을 깨끗한 폴더에서 다시 돌리고, 표 전체를 다른 기계에서 다시 만듭니다. 끝: 두 번째 실행 둘 다 14줄 모든 숫자가 같습니다." }, { "file": "corvel-comparison/studies/compare/collect.py", "location": "lines 120–121, cycles_per_token(opengemm8)", "problem": "Integer division truncates 136195 cycles over two tokens to 68097 rather than 68097.5. Both OpenGeMM M1 values are overstated by approximately 0.000734%. This favours the comparator and does not reverse the metric outcome.", "effect": "a metric", "how_to_check": "Read cycles=136195 and tokens=2 in both OpenGeMM sim.log files; divide without flooring and recompute M1 using their raw area and timing reports.", "status": "fixed", "resolution": "Exact division in collect.py. The rounding favoured OpenGeMM by 0.0007 %; no metric outcome changes.", "status_kr": "고침", "resolution_kr": "collect.py에서 정확히 나눕니다. 내림은 OpenGeMM에 0.0007% 유리했고 결과는 바뀌지 않습니다." }, { "file": "corvel-comparison/studies/compare/power_run.py", "location": "DESIGNS entries for tpu8 and gemmini8; corresponding sim.log files", "problem": "TPU and Gemmini replay only token 0, while CV-GEMV, VTA, OpenGeMM and W8A8 replay tokens 0 and 1. Normalizing useful MACs does not make their switching workloads identical. The direction and size of any energy bias remain unmeasured; the page's claim that every engine replays the same real tokens is inaccurate.", "effect": "a metric", "how_to_check": "Compare +tok=0 and one-token MAC denominators with two-token entries and raw RESULT lines. Replay the identical token set for every engine and recompute M2.", "status": "disclosed, not re-measured", "resolution": "Page and rules now state that the TPU and Gemmini replay the first token and the others two. Their energy per useful MAC is about ten times CV-GEMV's, so the token set cannot change the outcome.", "status_kr": "밝힘, 다시 재지 않음", "resolution_kr": "TPU와 Gemmini는 첫 토큰, 나머지는 두 토큰을 재생한다고 페이지와 규칙에 적었습니다. 두 엔진의 유효 MAC당 에너지는 CV-GEMV의 약 열 배라 토큰 구성으로 결과가 바뀔 수 없습니다." }, { "file": "corvel-comparison/RULES.md", "location": "opening paragraph and section 7, lines 93–98; site/comparison-en.html, 'How the verdict is made'", "problem": "The assertion that post-result changes can only favour comparators conflicts with the documented r1-to-r7 subject optimization and process-specific subject selection. Both Nangate45 r7 settings meet timing, so 'the setting that meets the target clock' does not uniquely select PAIR_REG=0. That setting wins M1; PAIR_REG=1 loses. The explicit amended subject names are implemented correctly, but their selection is not demonstrated to be independent of results.", "effect": "page wording", "how_to_check": "Compare both Nangate45 r7 rows' met_target and M1 fields, the r1 results, and the dated amendment. Describe the result as applying to the amended, selected configurations; remove the comparator-only characterization.", "status": "fixed", "resolution": "Rules and page now say that two post-result changes concern CV-GEMV (r2 to r7, and PAIR_REG fixed per process after the first results; on Nangate45 both settings meet the clock and the smaller layout was chosen). The claim names the configuration.", "status_kr": "고침", "resolution_kr": "결과 뒤 변경 중 두 가지가 CV-GEMV에 관한 것(r2~r7, 처음 결과 뒤 공정마다 정한 PAIR_REG; Nangate45에서는 두 설정 모두 클록을 맞춰 더 작은 레이아웃을 고름)이라고 규칙과 페이지에 적었습니다. 주장 문장에 구성을 밝혔습니다." }, { "file": "site/comparison-en.html", "location": "line 1, 'Every engine computes the same integers'", "problem": "The W8A8 baseline uses separately quantized per-row INT8 weights, not the W4A8-G32 integers. Its PASS means agreement with its own reference. The baseline is labelled elsewhere, but this universal arithmetic statement is false.", "effect": "page wording", "how_to_check": "Read fixtures.py's W8A8 construction, fixtures.json.w8a8.weights and summary.json's W8A8 math field. Explicitly exempt that baseline from the identical-integer claim.", "status": "fixed", "resolution": "Page: every W4A8 engine computes the same integers; the W8A8 baseline is excepted.", "status_kr": "고침", "resolution_kr": "W4A8 엔진은 모두 같은 정수를 계산하고 W8A8 기준선은 예외라고 고쳤습니다." }, { "file": "corvel-comparison/studies/compare/published.py", "location": "lines 71–72; site/comparison-en.html, published-design scaling explanation", "problem": "The 4.9× energy ratio described as 'same RTL' uses Nangate45 PAIR_REG=0 and ASAP7 PAIR_REG=1. It therefore combines process changes with a different register configuration and cycle count. The arithmetic is reproducible, but this is not a controlled same-configuration process ratio.", "effect": "page wording", "how_to_check": "Inspect the two selected design keys and their PAIR_REG descriptions. Use matching configurations to isolate process scaling, or label the current ratio as selected-configuration scaling. Class C does not affect the verdict.", "status": "fixed", "resolution": "The scaling ratio now uses one configuration (PAIR_REG = 1) in both processes (5.6x).", "status_kr": "고침", "resolution_kr": "환산 비율을 두 공정에서 같은 설정(PAIR_REG = 1)으로 다시 계산했습니다(5.6배)." }, { "file": "site/comparison-en.html", "location": "line 1, 'Where the margin is small', accuracy bullet", "problem": "The parenthetical assigns +14.0% versus +14.1% in the opposite order to the sentence's CV-GEMV-versus-floating-scale comparison. CV-GEMV is +14.131%; floating-scale INT4 g32 is +13.974%. The accuracy table itself has the correct order.", "effect": "page wording", "how_to_check": "Compare formats.json.formats.int4_g32_corvel.delta_pct_all_projections with int4_g32_rtn.delta_pct_all_projections; reverse the parenthetical values.", "status": "fixed", "resolution": "Order corrected.", "status_kr": "고침", "resolution_kr": "순서를 바로잡았습니다." }, { "file": "corvel-comparison/README.md", "location": "'What is in the package and what is not'; collect.py traffic_bits", "problem": "CV-GEMV RTL, testbench, config.mk and integer fixtures are absent. Its M3 arithmetic reproduces 5.375 from read_count and assumed metadata reads, but the interface widths and metadata-read schedule cannot be independently checked against its implementation here. Likewise, identical utilization settings and cycle_count semantics cannot be fully audited for the subject. This is a disclosed evidence limit, not a demonstrated numerical error.", "effect": "none", "how_to_check": "Inspect the subject interface, metadata read enables, cycle counter and flow configuration when those artifacts are available. Public comparator interfaces support their stated operand widths and the disclosed host/scratchpad exclusions.", "status": "no change", "resolution": "CV-GEMV RTL, testbench and flow configs are supplied under an evaluation agreement; the limit is disclosed in the package README.", "status_kr": "바꾸지 않음", "resolution_kr": "CV-GEMV RTL·테스트벤치·흐름 설정은 평가 계약으로 제공합니다. 이 한계는 재현 묶음 README에 적혀 있습니다." }, { "file": "evidence/compare", "location": "per-row result.json.window_s; missing activity.saif files", "problem": "M2 can be reproduced from raw workload power and the supplied window_s, but the underlying SAIF DURATION and activity annotations cannot be independently audited because activity.saif is not included. Agreement with result.json does not independently establish that window extraction was correct.", "effect": "none", "how_to_check": "Publish the SAIF files or independently generated duration/activity evidence and compare DURATION times its time unit with window_s. Current calculations use the supplied windows, as requested.", "status": "fixed", "resolution": "Each row folder now has activity-header.txt: the SAIF header (timescale, duration) and the SHA-256 of the full SAIF.", "status_kr": "고침", "resolution_kr": "줄마다 activity-header.txt에 SAIF 머리(시간 단위, 길이)와 전체 SAIF의 SHA-256을 공개합니다." } ], "summary_note": "a salutation at the start of the reviewer's summary was removed" }, { "round": 2, "date": "2026-11-06", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/, the comparison pages", "instructions_sha256": "4e2b9af25fcbdb427971bc0d679a45b51e747a9d9cfd17e46a84cde6b14eb639", "conclusion": "confirm", "summary": "confirm under the amended rules. Independently recomputed all 30 rows; M1–M3 match throughout (units: GMAC/s/mm², pJ/MAC, bits/MAC). Both rerun tables contain all 14 required rows with every numeric field equal; all 14 recorded cross-machine netlist hash pairs match. Subject logs support G1–G3 and sampled M4a. Subject selection, comparator membership, best-row scoring, strict 1% thresholds and M4b comparisons are correct; M4b is 14.131% versus 22.254% and 25.655%. The OpenGeMM division and previously identified selection, W8A8, scaling and accuracy wording fixes are verified. Remaining findings do not overturn reached. Hash records were compared, not independently regenerated from netlists. No files were changed; verdict.py was inspected without execution.", "recomputed_rows": [ { "row": "nangate45-cv_gemv_r7c_p8", "m1": 459.32950735021643, "m2": 0.8334500847424959, "m3": 5.375, "matches_summary": true }, { "row": "asap7-cv_gemv_r7r_p8", "m1": 21849.064448237958, "m2": 0.17001999959271652, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-vta8_official", "m1": 447.00571731626104, "m2": 0.9562967602379937, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8_official", "m1": 20193.72594919453, "m2": 0.1918849503884452, "m3": 6, "matches_summary": true }, { "row": "nangate45-vta8", "m1": 443.3775066034954, "m2": 0.9864192272123983, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8", "m1": 17243.66931776312, "m2": 0.196943280027989, "m3": 6, "matches_summary": true }, { "row": "nangate45-opengemm8", "m1": 313.365889452936, "m2": 1.1491631235681978, "m3": 6, "matches_summary": true }, { "row": "asap7-opengemm8", "m1": 7544.929306601195, "m2": 0.23964751371225915, "m3": 6, "matches_summary": true }, { "row": "nangate45-w8a8_p8", "m1": 316.311264395914, "m2": 1.2649527814179615, "m3": 9, "matches_summary": true }, { "row": "asap7-w8a8_p8", "m1": 16489.799316150493, "m2": 0.23235266326445386, "m3": 9, "matches_summary": true } ], "findings": [ { "file": "evidence/compare/review.json; site/comparison-en.html", "location": "Round 1 finding 9, status/resolution; Independent review row 9", "problem": "The claimed fix is incomplete: activity-header.txt is absent for OpenGeMM and Gemmini in both processes. The other 26 headers have 1 ps timescales and durations matching result.json.window_s. For the four missing headers, M2 reproduces using window_s, but its duration cannot be independently checked against a supplied SAIF header. Headers also do not expose full switching activity.", "effect": "page wording", "how_to_check": "Check evidence/compare/{nangate45,asap7}-{opengemm8,gemmini8}/activity-header.txt. Supply those headers or describe the resolution as partial. No numerical discrepancy was demonstrated.", "status": "fixed", "resolution": "OpenGeMM and Gemmini were first replayed on the build VM, whose SAIF was not kept. Their activity-header.txt now comes from the clean rerun of the same routed netlist (whose numbers equal the first run's), and the file says so.", "status_kr": "고침", "resolution_kr": "OpenGeMM·Gemmini는 처음에 빌드 VM에서 재생했고 그 SAIF를 남기지 않았습니다. 같은 라우팅 넷리스트의 깨끗한 재실행(숫자가 처음 실행과 같음)에서 머리를 가져왔고 파일에 그렇게 적었습니다." }, { "file": "corvel-comparison/studies/compare/verdict.py", "location": "Lines 35–37 and 80–88: RERUN_KEYS and compare_run", "problem": "G4 checks only a whitelist, despite stating that every number must match. Numeric fields including die_area_um2, clock_used_mhz, useful_mac_per_s, utilization and the secondary metrics are omitted. This is a latent validation gap: independently comparing all supplied row fields found no differences, so it does not invalidate this verdict.", "effect": "none", "how_to_check": "Compare every numeric field for all 14 required rows in summary.json against both rerun tables. An in-memory change to an omitted field would escape the current comparison.", "status": "fixed", "resolution": "G4 now compares every scalar field of every needed row, not a list of fields.", "status_kr": "고침", "resolution_kr": "G4가 이제 필요한 줄의 모든 숫자 칸을 비교합니다." }, { "file": "corvel-comparison/README.md", "location": "Step 5 and 'Recomputing the table from the published raw files only'", "problem": "The documented verdict commands omit --second-machine. With the revised code, these commands necessarily fail G4 and cannot reproduce the published reached verdict. The description that verdict.py reads only summary.json and formats.json is insufficient for a passing G4.", "effect": "none", "how_to_check": "Read verdict.py lines 90–96. Reproduction instructions must provide --second-machine pointing to second-machine-summary.json and, when checking the additional clean run, --rerun pointing to rerun-summary.json. I did not execute verdict.py because it writes verdict.json.", "status": "fixed", "resolution": "The package README gives the verdict command with --second-machine and --rerun.", "status_kr": "고침", "resolution_kr": "재현 묶음 README의 판정 명령에 --second-machine과 --rerun을 넣었습니다." }, { "file": "corvel-comparison/studies/compare/power_run.py", "location": "DESIGNS entries for tpu8 and gemmini8; corresponding sim.log files", "problem": "The round-1 workload difference remains: TPU and Gemmini replay token 0 only, whereas the other engines replay two tokens. Their M2 values therefore describe different activity samples. This is now explicitly disclosed and permitted by the dated rules; it is not a newly discovered arithmetic error or a demonstrated reversal.", "effect": "a metric", "how_to_check": "Compare token selections, replay logs and useful_macs_in_window. Replay identical token sets to measure the effect on M2.", "status": "re-measured", "resolution": "The second token was replayed on the TPU and Gemmini netlists in both processes; energy per useful MAC changes by at most 0.04 % (token-check.json).", "status_kr": "다시 잼", "resolution_kr": "두 공정의 TPU·Gemmini 넷리스트에 둘째 토큰을 재생했고, 유효 MAC당 에너지 차이는 최대 0.04%입니다(token-check.json)." }, { "file": "corvel-comparison/README.md", "location": "'What is in the package and what is not'; subject rows' M3_basis", "problem": "The disclosed round-1 evidence limitation remains: CV-GEMV RTL, testbench and config.mk are absent. Its 5.375-bit M3 reproduces from logged read_count and the stated metadata schedule, but the actual interface, metadata enables, cycle-counter semantics and utilization configuration cannot be independently inspected here. Public comparator interfaces support their stated bases and exclusions.", "effect": "none", "how_to_check": "Inspect the subject artifacts supplied under the evaluation agreement. Current subject M3 agreement verifies the stated accounting, not its hidden implementation.", "status": "no change", "resolution": "CV-GEMV RTL, testbench and flow configs are supplied under an evaluation agreement; disclosed.", "status_kr": "바꾸지 않음", "resolution_kr": "CV-GEMV RTL·테스트벤치·흐름 설정은 평가 계약으로 제공하며, 이 한계는 밝혀 두었습니다." } ], "summary_note": "a salutation at the start of the reviewer's summary was removed" }, { "round": 3, "date": "2026-11-07", "revision": "r11", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/, before-13/summary.json, the comparison pages", "instructions_sha256": "0edbbd96c675fe79ee4f1fecb1333ce749eaec61b0032770678854c2dbcd2a8a", "conclusion": "confirm", "conclusion_main": "confirm", "conclusion_open_pdk": "confirm", "summary": "both numerical conclusions stand: main reached; open PDKs not reached. All 11 requested unique rows reproduce without collect.py (M1 in GMAC/s/mm², M2 in pJ/MAC). Subject selection, four-row VTA best scoring and strict ±1% comparisons match the rules. Executing verdict.py with writes intercepted reproduces verdict.json exactly. G4 independently matches every scalar across 18 main second-machine rows, 18 clean-rerun rows and 27 open-PDK rows. All 74 earlier rows are entirely unchanged. M4a logs and M4b perplexity arithmetic support their passes. GF180 revision minima reproduce 10.90, 9.37, 8.05, 9.15 and 8.48 ns. The page discloses the retrospective main-rule change and correctly keeps current-main status pending review. VTA's three edits preserve pipeline placement and valid arithmetic; an independent exhaustive check of the Booth expression passes all 8,192 operand pairs. No functional weakening was identified. Missing lock-step source limits coverage verification. IHP's vacuous G2 acceptance is unjustified, but correcting it leaves the open-PDK failure unchanged. No flows or model inference were rerun.", "recomputed_rows": [ { "row": "nangate45-cv_gemv_r11c_p8", "m1": 617.4969996520294, "m2": 0.6972288804683828, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-vta8_tuned", "m1": 472.89153899031425, "m2": 0.9654077533302021, "m3": 6, "matches_summary": true }, { "row": "nangate45-vta8_official", "m1": 447.00571731626104, "m2": 0.9562967602379937, "m3": 6, "matches_summary": true }, { "row": "asap7-cv_gemv_r11r_p8", "m1": 27801.30753024478, "m2": 0.15034179155656627, "m3": 5.375, "matches_summary": true }, { "row": "asap7-vta8_official_tuned", "m1": 22153.895738143186, "m2": 0.19015738049795966, "m3": 6, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r11c_p8", "m1": 56.410063119195826, "m2": 3.756434176245668, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd-vta8_tuned", "m1": 45.781880091401696, "m2": 5.185081934326837, "m3": 6, "matches_summary": true }, { "row": "gf180-cv_gemv_r11r_p8", "m1": 10.255207778804426, "m2": 110.65859926262188, "m3": 5.375, "matches_summary": true }, { "row": "gf180-vta8_official_tuned", "m1": 6.878411200150909, "m2": 121.70136572629494, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r11c_p8", "m1": 27.5100461015547, "m2": 3.486503401760015, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2-vta8_tuned", "m1": 22.655881894632167, "m2": 5.010422379528716, "m3": 6, "matches_summary": true } ], "findings": [ { "file": "corvel-comparison/studies/compare/verdict.py", "location": "lines 75–78; verdict.json open_pdk.gates.G2 ihp-sg13g2", "problem": "The code accepts vacuous equivalence and cites a G2 exception absent from RULES.md. The IHP subject's lec.txt explicitly reports no valid outputs to compare. G1 replay cannot establish the formal equivalence required by G2. G2 should fail or remain unestablished. This does not change the current open-PDK 'not reached' outcome, already forced by GF180 timing.", "effect": "none", "how_to_check": "Compare RULES.md section 4 with evidence/compare/ihp-sg13g2-cv_gemv_r11c_p8/lec.txt and the vacuous branch in opdk_verdict.", "status": "fixed", "resolution": "G2 in the open-PDK verdict now marks a vacuous equivalence check as open (pass: false) and notes on the page that G1 real-token replay passes.", "status_kr": "고침", "resolution_kr": "오픈 PDK 판정의 G2가 이제 비교할 출력을 찾지 못한 동치 검사를 열림(pass: false)으로 두고, 페이지에 G1 실제 토큰 재생이 통과한다고 적습니다." }, { "file": "evidence/compare/vta-tuned/summary.json", "location": "checks.lockstep.what; referenced vta/tuned/lockstep.sh", "problem": "The named lockstep.sh is absent from this folder. The six summary log lines report zero mismatches, but without the driver its randomization, extreme-value stretches, reset coverage and comparison predicates cannot be independently inspected. The supplied real-layer testbench and four matching replay logs support unchanged results on those inputs, not exhaustive sequential equivalence.", "effect": "none", "how_to_check": "Inventory studies/compare/vta/tuned and search the package for lockstep. Supply the exact driver/testbench used for lockstep.txt, then rerun its three seeds against each original/tuned pair.", "status": "fixed", "resolution": "make_package.py now includes vta/tuned/lockstep.sh and the generated w5 official VTA module in reproduce.zip.", "status_kr": "고침", "resolution_kr": "make_package.py가 이제 vta/tuned/lockstep.sh와 w5 공식 VTA 생성 모듈을 reproduce.zip에 넣습니다." }, { "file": "corvel-comparison/studies/compare/rtl/vta_mvm_port_tuned.v", "location": "lines 72–81, s0/s1/dot reduction; RULES.md r10/r11 additions", "problem": "The three advertised VTA edits are present and mathematically sound, but they do not establish exhaustive optimization parity. VTA still uses cascaded carry-propagating additions for its multioperand dot-product reduction; CV-GEMV received carry-save arithmetic. A carry-save reduction is also applicable to VTA's combinational reduction without adding pipeline stages. Its benefit or harm has not been measured. Conversely, CV-GEMV's group-scale sharing and waiting-group changes have no direct counterpart in this VTA core.", "effect": "page wording", "how_to_check": "Compare the VTA adder tree with RULES.md's r10/r11 arithmetic changes. Describe the tuning as the three specific changes actually measured; any stronger parity claim requires an additional equivalent VTA implementation and measurements.", "status": "fixed", "resolution": "The page and vta-tuned/summary.json now state that VTA received the three changes from r9 and r10 that VTA's structure has, and that VTA's combinational dot-product adder tree is unchanged.", "status_kr": "고침", "resolution_kr": "페이지와 vta-tuned/summary.json에 VTA 구조에 해당하는 r9·r10 변경 세 가지를 적용했고 조합 내적 덧셈 트리는 그대로라고 밝혔습니다." }, { "file": "evidence/compare/vta-tuned/summary.json", "location": "what; changes; corresponding site/comparison-en.html engine description", "problem": "'Registers ... unchanged' is imprecise: register positions and pipeline depth remain unchanged, but registered product widths shrink and accumulator registers update during invalid cycles. Valid-result behavior is preserved by the inspected logic; internal state and invalid-cycle data are not identical.", "effect": "page wording", "how_to_check": "Inspect PipeAdder rA/rB width changes in generated-module.diff and removal of the two accumulator load enables in both diffs. Say 'pipeline stages and valid-result latency unchanged'.", "status": "fixed", "resolution": "Changed to 'pipeline stages, valid-result latency and results unchanged' on the page and in vta-tuned/summary.json.", "status_kr": "고침", "resolution_kr": "페이지와 vta-tuned/summary.json의 문구를 '파이프라인 단계·유효 결과 지연·결과는 그대로'로 바꿨습니다." }, { "file": "corvel-comparison/studies/compare/verdict.py", "location": "opdk_verdict lines 102–106", "problem": "The open-PDK verdict says M4b carries over, but its outcome never incorporates the main M4b result. A future format-accuracy failure could therefore leave the open-PDK code verdict reached. All supplied M4b comparisons pass, so this is a latent validation defect, not a current reversal.", "effect": "none", "how_to_check": "Trace opdk_verdict's metric list: it contains only M1–M3. In an in-memory test, make CV-GEMV's accuracy delta worse than both baselines and observe that the open-PDK computation does not consult it.", "status": "fixed", "resolution": "opdk_verdict now includes M4b in its metric pass check.", "status_kr": "고침", "resolution_kr": "opdk_verdict가 이제 지표 통과 판정에 M4b도 포함합니다." }, { "file": "site/comparison-en.html", "location": "line 1, Attempts paragraph and main-process tables", "problem": "'All of them stay in the table' is true of summary.json, but the rendered tables omit six earlier CV-GEMV configurations per main process: r2, r2d, r3d, r4d, r5d and r6d. No evidence rows were deleted; the page should distinguish its displayed subset from the complete downloadable table.", "effect": "page wording", "how_to_check": "Compare summary.json row keys with the page's per-row sim.log links. All 74 historical rows remain field-for-field identical in the 104-row summary.", "status": "fixed", "resolution": "The page now says that all of them stay in summary.json and that the tables on the page show r1 and r7 to r11.", "status_kr": "고침", "resolution_kr": "페이지에 모든 판을 summary.json에 남기고 이 페이지 표에는 r1과 r7~r11을 표시한다고 적었습니다." }, { "file": "site/comparison-en.html", "location": "line 1, History paragraph", "problem": "The earlier r7 verdict is described as 'confirmed by two reviews', but the page's own review record says round 1 disagreed and round 2 confirmed after corrections.", "effect": "page wording", "how_to_check": "Compare the introductory history sentence with the round-1 and round-2 conclusions in the Independent review section and review.json.", "status": "fixed", "resolution": "Changed to 'confirmed in round 2 after round 1's findings were fixed'.", "status_kr": "고침", "resolution_kr": "'1차 지적을 고친 뒤 2차 검토에서 확인됐습니다'로 바꿨습니다." }, { "file": "corvel-comparison/README.md", "location": "Recomputing the table: verdict command", "problem": "The documented command omits --opdk-second-machine, so it cannot reproduce the published open-PDK G4 pass or the complete verdict.json. Several surrounding descriptions also still identify r7 as the current subject.", "effect": "none", "how_to_check": "Run verdict.py with output intercepted in memory, first using the documented arguments and then adding --opdk-second-machine evidence/compare/opdk-second-machine-summary.json. The latter reproduces the supplied verdict.json exactly.", "status": "fixed", "resolution": "The package README now includes --opdk-second-machine in the verdict commands and updates the revision descriptions through r11.", "status_kr": "고침", "resolution_kr": "재현 묶음 README의 판정 명령에 --opdk-second-machine을 넣고 설명을 r11까지 맞췄습니다." }, { "file": "corvel-comparison/RULES.md", "location": "introductory paragraph; site/comparison-en.html 'How the verdict is made'", "problem": "The introductory accounting of post-result changes is stale: RULES.md mentions subject development only through r7, while the page's closing explanation stops at r10. The dated additions and open-PDK history correctly describe r11. The supplied dated prose alone does not independently prove the claim that each rule was committed before its result.", "effect": "page wording", "how_to_check": "Reconcile the introductory descriptions with the dated r8–r11 additions. To substantiate commit chronology, provide the corresponding version-control commits and result-publication records.", "status": "fixed", "resolution": "RULES.md and the page's 'How the verdict is made' section now both state r2 to r11 and the 2026-11-07 subject-rule amendment.", "status_kr": "고침", "resolution_kr": "RULES.md와 페이지의 '판정 방법' 모두 r2~r11과 2026-11-07 대상 규칙 변경을 적도록 맞췄습니다." }, { "file": "evidence/compare/summary.json", "location": "CV-GEMV M3_basis and cycles_basis", "problem": "The disclosed implementation-evidence limit remains: CV-GEMV RTL and its testbench are unavailable. Its M3 reproduces as (68096×320 + 152×112×8×12)/(896×4864) = 5.375, but the actual metadata-read schedule and counter semantics cannot be independently inspected. VTA's public interface and replay support 5 + 8/8 = 6 bits per useful MAC, with host/storage exclusions as written.", "effect": "none", "how_to_check": "Compare sim.log read counts and the stated M3 basis; implementation-level confirmation requires the withheld CV-GEMV interface and testbench.", "status": "no change", "resolution": "CV-GEMV RTL, testbench and flow configs are supplied under an evaluation agreement; disclosed.", "status_kr": "바꾸지 않음", "resolution_kr": "CV-GEMV RTL·테스트벤치·흐름 설정은 평가 계약으로 제공하며, 이 한계는 밝혀 두었습니다." } ], "summary_note": "a salutation at the start of the reviewer's summary was removed" }, { "round": 4, "date": "2026-11-07", "revision": "r11", "vta_rows": 6, "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/, before-csa/summary.json, the comparison pages", "instructions_sha256": "31030d986922c73a4d7a58b3fae6528f1dad04159e9e0e9a3b40a5543f57b9f7", "conclusion": "confirm", "conclusion_main": "confirm", "conclusion_open_pdk": "confirm", "summary": "both final outcomes stand: main reached; open-PDK not reached because GF180 misses timing by 0.484495 ns. All 15 independently recomputed rows match; units are GMAC/s/mm², pJ/MAC and bits/MAC. Running verdict.py with writes intercepted reproduced verdict.json exactly. Six-row VTA selection and strict ±1% thresholds are correct; required second-machine coverage is 22/22 and 33/33 with identical scalars. All 104 earlier rows remain unchanged except permitted IHP equivalence fields; exactly ten CSA rows were added. Both diffs and source hashes match. Exhaustive independent arithmetic evaluation covers all 8,192 multiplier inputs; compressor and bias identities establish row arithmetic modulo 2^32. Registers and valid latency are preserved. No extra reduction level, live legacy adder path, or further directly applicable r8–r11 omission was identified; exhaustive optimization parity is not established. Published lock-step and real-layer logs report zero mismatches. Main margins (+29.9%, +18.4%; energy −24.3%, −16.2%) and pending-review status are correct. IHP output-collection evidence and open-PDK clean-rerun coverage remain incomplete as detailed above. No files were changed.", "summary_note": "a salutation at the start of the reviewer's summary was removed", "recomputed_rows": [ { "row": "nangate45-cv_gemv_r11c_p8", "m1": 617.496999652, "m2": 0.697228880468, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-vta8_csa", "m1": 475.216079203, "m2": 0.920621568929, "m3": 6, "matches_summary": true }, { "row": "nangate45-vta8_official_csa", "m1": 468.162028606, "m2": 0.9226629014, "m3": 6, "matches_summary": true }, { "row": "asap7-cv_gemv_r11r_p8", "m1": 27801.307530245, "m2": 0.150341791557, "m3": 5.375, "matches_summary": true }, { "row": "asap7-vta8_csa", "m1": 23478.483914866, "m2": 0.179309754147, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8_official_csa", "m1": 23123.637460907, "m2": 0.179891965201, "m3": 6, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r11c_p8", "m1": 56.410063119, "m2": 3.756434176246, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd-vta8_csa", "m1": 45.558714773, "m2": 5.077658845325, "m3": 6, "matches_summary": true }, { "row": "sky130hd-vta8_official_csa", "m1": 47.251586026, "m2": 4.877263251004, "m3": 6, "matches_summary": true }, { "row": "gf180-cv_gemv_r11r_p8", "m1": 10.255207779, "m2": 110.658599262622, "m3": 5.375, "matches_summary": true }, { "row": "gf180-vta8_csa", "m1": 6.79401013, "m2": 118.130608997324, "m3": 6, "matches_summary": true }, { "row": "gf180-vta8_official_csa", "m1": 7.204240313, "m2": 114.736359397778, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r11c_p8", "m1": 27.510046102, "m2": 3.48650340176, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2-vta8_csa", "m1": 23.304656107, "m2": 4.935926870528, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2-vta8_official_csa", "m1": 22.95953472, "m2": 4.977539801345, "m3": 6, "matches_summary": true } ], "findings": [ { "file": "evidence/compare/ihp-sg13g2-cv_gemv_r11c_p8/lec.txt", "location": "lines 1–2; corresponding IHP lec.txt files", "problem": "The published corrected logs contain only 'SAT solver finished: UNSAT' and 'Circuits are IDENTICAL'. They omit collected-output counts and database/netlist provenance. The scripts correctly invoke the flow's writer and checker on the row's base databases with only bondpad_70* removed, but these excerpts do not independently establish the requested non-vacuous output collection. This is an evidence gap, not evidence that the corrected comparison was vacuous.", "effect": "none", "how_to_check": "Publish the complete corrected checker and netlist-writing logs from both machines, including positive output counts and input database/netlist identities. The open-PDK outcome remains not reached because GF180 fails G3.", "status": "fixed", "resolution": "lec.txt of every IHP SG13G2 row now also lists the databases read, the two netlists compared and the collected input and output counts (2,825 outputs for the r11c subject); all 36 IHP rows were re-run on both machines with that record, all identical.", "status_kr": "고침", "resolution_kr": "모든 IHP SG13G2 줄의 lec.txt에 읽은 데이터베이스, 비교한 두 넷리스트, 모은 입력·출력 개수(판정 대상 r11c는 출력 2,825개)를 함께 적습니다. 두 기계의 IHP 36줄을 그 기록으로 다시 돌렸고 모두 동일입니다." }, { "file": "corvel-comparison/studies/compare/verdict.py", "location": "opdk_verdict, lines 92–101; evidence/compare/rerun-summary.json", "problem": "Open-PDK G4 checks the second-machine table but never checks the supplied clean rerun. The published clean-rerun table contains only Nangate45/ASAP7, so it does not substantiate the 08:33 addition's statement that the new rows were measured on the clean rerun in all five processes. All 33 required open-PDK rows are present and scalar-identical on the second machine.", "effect": "none", "how_to_check": "Supply the open-PDK clean-rerun rows and apply the same completeness/scalar-equality check to them. This cannot reverse the existing open-PDK not-reached outcome.", "status": "fixed", "resolution": "The public text of the 08:33 addition now carries the parenthesis of the internal original written at the same time: the clean rerun covers Nangate45 and ASAP7. The open-PDK G4 uses the second machine, as its rule says.", "status_kr": "고침", "resolution_kr": "08:33 조항의 공개본에, 같은 시각에 쓴 내부 원문의 괄호(깨끗한 재실행은 Nangate45·ASAP7)를 되살렸습니다. 오픈 PDK G4는 규칙대로 두 번째 기계를 봅니다." }, { "file": "site/comparison-en.html", "location": "open-PDK IHP paragraph; corvel-comparison/RULES.md 09:13 addition", "problem": "The assertion that the original flow compared empty netlists 'for every engine' exceeds the supplied records: ihp-sg13g2-tpu8/lec-flow.txt says 'no final LEC log'. The other 19 IHP flow excerpts explicitly report vacuous equivalence.", "effect": "page wording", "how_to_check": "Qualify the statement to distinguish observed vacuous checks from the TPU's missing original check, or supply its original full log.", "status": "fixed", "resolution": "The page and both rule files now say: every IHP row whose flow ran the check compared empty netlists; the TPU's flow left no final check log.", "status_kr": "고침", "resolution_kr": "페이지와 두 규칙 파일이 이제 '흐름이 그 검사를 돌린 IHP 줄 전부가 빈 넷리스트를 비교했고, TPU는 흐름의 검사 기록이 없다'고 적습니다." }, { "file": "site/comparison-en.html", "location": "open-PDK introduction: 'rules fixed before any result'", "problem": "This unqualified statement conflicts with the disclosed later subject, comparator, G2 and second-machine amendments. The initial open-PDK extension preceded results; the complete current rules did not.", "effect": "page wording", "how_to_check": "Describe the initial conditions as fixed before open-PDK results and explicitly acknowledge the dated amendments.", "status": "fixed", "resolution": "The open-PDK section now says its conditions were fixed before the first open-PDK result and that later changes are dated in the rules.", "status_kr": "고침", "resolution_kr": "오픈 PDK 절이 이제 '조건은 첫 오픈 PDK 결과 전에 정했고 이후 변경은 규칙에 날짜와 함께 적었다'고 씁니다." }, { "file": "corvel-comparison/studies/compare/vta/tuned/lockstep.sh", "location": "generated stimulus: all three input-valid ports connected to v", "problem": "The lock-step stimulus exercises common valid gaps, not independently varying input-valid signals, and resets rst only initially. Its six passing runs support sampled valid-result equivalence, not exhaustive sequential equivalence. Inspection shows the relevant valid/reset logic is unchanged, so this does not reveal a functional defect.", "effect": "none", "how_to_check": "For stronger regression coverage, vary the three valid inputs independently and exercise midstream rst; retain the arithmetic identity argument separately from simulation coverage.", "status": "no change", "resolution": "Kept as stated: the lock-step checks common valid gaps and the initial reset; the arithmetic identity is argued separately. A stimulus with independent valid inputs and midstream reset is a follow-up.", "status_kr": "그대로", "resolution_kr": "밝힌 대로 둡니다. 맞대기는 공통 유효 신호 공백과 처음 리셋을 봅니다. 산술 항등식은 따로 논증합니다. 유효 입력을 따로 흔들고 중간 리셋을 넣는 자극은 후속 과제입니다." }, { "file": "evidence/compare/summary.json", "location": "CV-GEMV M3_basis and sources", "problem": "M3 reproduces as 5.375 from tb_read_count and the stated metadata accounting, but private CV-GEMV RTL/testbench prevents independent inspection of the actual metadata-read schedule. This inherited evidence limit remains unchanged.", "effect": "none", "how_to_check": "Inspect the private read interface and testbench counter semantics under evaluation access; the public arithmetic is (68096×320 + 152×112×8×12)/(896×4864).", "status": "no change", "resolution": "Inherited, disclosed limit: CV-GEMV RTL, testbench and flow configs are under an evaluation agreement.", "status_kr": "그대로", "resolution_kr": "이어 온 공개 한계입니다. CV-GEMV RTL·테스트벤치·흐름 설정은 평가 계약으로 제공합니다." } ] }, { "round": 5, "date": "2026-11-07", "revision": "r11", "vta_rows": 6, "scope": "flow sensitivity", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (sensitivity.json, sensitivity/), the comparison pages", "instructions_sha256": "07961c299c48ba20ab6bc23cd3f4a06e261b820b55d8f5afb927a579aab80bd5", "conclusion": "disagree", "row_selection_ok": true, "summary": "The numerical study checks out; the determinism wording needs correction. All 21 selected rows qualify, with none missing or extra. All 90 variant-row setting logs match the intended utilization and density add-on. ASAP7's base utilization is 50%, and all numeric fields of its three u50 rows equal the published rows. Every margin, best-row name and smallest value recomputes correctly, including the separately reported added pair. All 30 requested raw-file M1/M2 checks pass within 0.1%; M1 is in GMAC/s/mm² and M2 in pJ/useful MAC. Replay durations were independently read from activity headers and power from workload-driven totals, without collect.py. The closest rule margin is GF180MCU M2 at -2.914067% under d015. Neither metric reverses. Both pages plainly disclose the ASAP7 correction, separate a45/a55 from the rule's minimum, and preserve the failed open-PDK timing gate and unchanged verdicts. The statement that clock/DRC/replay status stays unchanged holds for the four rule variants; it must remain scoped to them because added a55 makes ASAP7 vta8_csa miss timing by 0.30 ps. The asymmetric original ASAP7 utilization sweep is disclosed and supplemented by 45/55%; it provides no hidden numerical advantage in the reported calculations. No files were modified.", "summary_note": "a salutation at the start of the reviewer's summary was removed", "recomputed_rows": [ { "variant": "u40", "row": "gf180-cv_gemv_r11r_p8", "m1": 10.159752, "m2": 110.291687, "matches_table": true }, { "variant": "u40", "row": "gf180-vta8_official_tuned", "m1": 6.844791, "m2": 122.240287, "matches_table": true }, { "variant": "u40", "row": "gf180-vta8_csa", "m1": 6.802155, "m2": 119.883667, "matches_table": true }, { "variant": "u40", "row": "gf180-vta8_official_csa", "m1": 7.012067, "m2": 114.68412, "matches_table": true }, { "variant": "u50", "row": "gf180-cv_gemv_r11r_p8", "m1": 10.190519, "m2": 109.006546, "matches_table": true }, { "variant": "u50", "row": "gf180-vta8_official_tuned", "m1": 6.837433, "m2": 121.037414, "matches_table": true }, { "variant": "u50", "row": "gf180-vta8_csa", "m1": 6.812119, "m2": 118.527763, "matches_table": true }, { "variant": "u50", "row": "gf180-vta8_official_csa", "m1": 7.21533, "m2": 112.643487, "matches_table": true }, { "variant": "d015", "row": "gf180-cv_gemv_r11r_p8", "m1": 10.165202, "m2": 111.02635, "matches_table": true }, { "variant": "d015", "row": "gf180-vta8_official_tuned", "m1": 6.820354, "m2": 121.506795, "matches_table": true }, { "variant": "d015", "row": "gf180-vta8_csa", "m1": 6.85921, "m2": 118.971018, "matches_table": true }, { "variant": "d015", "row": "gf180-vta8_official_csa", "m1": 7.008711, "m2": 114.358844, "matches_table": true }, { "variant": "d025", "row": "gf180-cv_gemv_r11r_p8", "m1": 10.206139, "m2": 110.535541, "matches_table": true }, { "variant": "d025", "row": "gf180-vta8_official_tuned", "m1": 6.860954, "m2": 120.872245, "matches_table": true }, { "variant": "d025", "row": "gf180-vta8_csa", "m1": 6.76581, "m2": 118.757897, "matches_table": true }, { "variant": "d025", "row": "gf180-vta8_official_csa", "m1": 7.196379, "m2": 113.870449, "matches_table": true }, { "variant": "u40", "row": "nangate45-cv_gemv_r11c_p8", "m1": 616.51837, "m2": 0.698626, "matches_table": true }, { "variant": "u50", "row": "nangate45-cv_gemv_r11c_p8", "m1": 617.803549, "m2": 0.694047, "matches_table": true }, { "variant": "d015", "row": "nangate45-cv_gemv_r11c_p8", "m1": 617.237604, "m2": 0.700438, "matches_table": true }, { "variant": "d025", "row": "nangate45-cv_gemv_r11c_p8", "m1": 617.462406, "m2": 0.696133, "matches_table": true }, { "variant": "u40", "row": "asap7-cv_gemv_r11r_p8", "m1": 27729.837884, "m2": 0.152094, "matches_table": true }, { "variant": "u50", "row": "asap7-cv_gemv_r11r_p8", "m1": 27801.709196, "m2": 0.150342, "matches_table": true }, { "variant": "d015", "row": "asap7-cv_gemv_r11r_p8", "m1": 27822.347462, "m2": 0.15005, "matches_table": true }, { "variant": "d025", "row": "asap7-cv_gemv_r11r_p8", "m1": 27799.377034, "m2": 0.150261, "matches_table": true }, { "variant": "a45", "row": "asap7-cv_gemv_r11r_p8", "m1": 27777.179719, "m2": 0.151131, "matches_table": true }, { "variant": "a45", "row": "asap7-vta8_csa", "m1": 23389.759581, "m2": 0.180797, "matches_table": true }, { "variant": "a45", "row": "asap7-vta8_official_csa", "m1": 22990.645603, "m2": 0.181919, "matches_table": true }, { "variant": "a55", "row": "asap7-cv_gemv_r11r_p8", "m1": 27820.011836, "m2": 0.150219, "matches_table": true }, { "variant": "a55", "row": "asap7-vta8_csa", "m1": 23536.633498, "m2": 0.178696, "matches_table": true }, { "variant": "a55", "row": "asap7-vta8_official_csa", "m1": 23244.998679, "m2": 0.17906, "matches_table": true } ], "findings": [ { "file": "site/comparison-en.html; site/comparison-kr.html", "location": "#sensitivity, opening determinism sentence", "problem": "The claim of the same routed netlist on the build machine, Mac and AWS exceeds the supplied evidence. second-machine.json records each row against either Mac or AWS, not both. For example, ASAP7 vta8_csa and vta8_official_csa have AWS comparisons, while the ASAP7 subject has a Mac comparison. G4 establishes scalar-table equality on the required second machine; it does not establish three-machine netlist identity or universal determinism.", "effect": "page wording", "how_to_check": "Group second-machine.json rows by platform/design and inspect second_machine and netlist hashes. Compare this coverage with verdict.json gates.G4. Say that the supplied reruns reproduced the table and recorded netlist hashes on each row's designated second machine.", "status": "fixed", "status_kr": "고침", "resolution": "The opening sentence now says what was checked: one second machine per row (the Mac or AWS) reproduced every number in the table (G4), with the same routed netlist for 22 of 22 main-verdict rows, and the three ASAP7 rows run again at the published setting reproduced every number on AWS. The clock, DRC and replay sentence is limited to the rule's four variants, and the added 55 % run's VTA hand-port row that misses 1000 ps by 0.30 ps is stated under the added table.", "resolution_kr": "첫 문장을 확인한 만큼만 말하도록 고쳤습니다. 줄마다 두 번째 기계 하나(맥 또는 AWS)가 표의 모든 숫자를 재현했고(G4), 주 판정 22줄 모두 라우팅 넷리스트가 같았으며, 공개 설정으로 다시 돌린 ASAP7 세 줄이 AWS에서 모든 숫자를 재현했다는 내용입니다. 클록·DRC·재생 문장은 규칙의 네 변형으로 한정했고, 추가 55% 실행에서 VTA 손 이식판 줄이 1000 ps를 0.30 ps 차이로 못 맞춘 것을 추가 표 아래에 적었습니다." }, { "file": "corvel-comparison/RULES.md", "location": "2026-11-07 14:34 addition, baseline-based 5% VTA selection", "problem": "Selecting competitors using only baseline performance can favour CV-GEMV if an excluded VTA row improves more under a changed setting. For example, SKY130HD vta8_official_tuned is excluded with M1 only 5.2071% below the baseline best and M2 7.4794% above it. Its variant performance is unmeasured. This is a limitation of the selected-row study, not an incorrect application of its rule or a demonstrated reversal.", "effect": "none", "how_to_check": "Derive eligibility from summary.json and then run the excluded VTA rows under the same variants to test whether any becomes best. Choosing the best included VTA separately for each metric and variant is conservative toward VTA and introduces no additional favourable selection for CV-GEMV.", "status": "fixed", "status_kr": "고침", "resolution": "The 14 VTA rows the 14:34 rule left out were run under the same four variants, and ASAP7's four of them also at 45 and 55 % (rules, 2026-11-07 18:48, committed before the runs). Against the best of all six VTA rows no metric reverses or comes within 1 % in any variant, and no left-out row is VTA's best, so the margins equal the rule's (smallest GF180MCU M2 -2.9 %, density add-on 0.15). Both pages report this apart from the rule's result, and the limit paragraph is gone. Round 7 checked it.", "resolution_kr": "14:34 규칙이 뺀 VTA 14줄을 같은 네 변형으로, 그중 ASAP7 네 줄은 45%·55%로도 돌렸습니다(규칙 2026-11-07 18:48, 실행 전에 커밋). VTA 여섯 줄 전체의 최고값과 비교해도 어느 변형에서나 뒤집히거나 1% 안으로 들어온 지표가 없고, 빠진 줄이 VTA 최고가 된 곳도 없어 우위는 규칙 결과와 같습니다(가장 작은 값 GF180MCU M2 -2.9%, 밀도 여유 0.15). 두 페이지에 규칙 결과와 따로 적었고 한계 문단은 없앴습니다. 7차 교차 검토가 확인했습니다." } ] }, { "round": 6, "date": "2026-11-07", "revision": "r11", "vta_rows": 6, "scope": "flow sensitivity", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (sensitivity.json, sensitivity/, review.json round 5), the comparison pages", "instructions_sha256": "fecb717bdb59a8f387a1d12adf519bd6f5fb982e79f2850ca2b6a02e87f3fd64", "conclusion": "confirm", "row_selection_ok": true, "summary": "The revised wording holds in both languages. All 22 recorded main-verdict netlists match their designated second machine (20 Mac, 2 AWS); G4 reports complete scalar reproduction for 22 main and 33 open-PDK rows. All three ASAP7 u50 rows are identical_to_published. Clock/DRC/replay invariance correctly applies only to the four rule variants. Added a55 correctly identifies vta8_csa missing 1000 ps by 0.299982 ps, rounded to 0.30 ps. Excluded-row performance is plainly stated as unmeasured, and round 5 retains its disagree conclusion and open limitation. All 21 selected rows satisfy the rule. The four checked margins reproduce: GF180MCU M2 d015 −2.914067%, d025 −2.928686%; ASAP7 M1 u40 +19.679409%, a55 +18.197065%. Raw recomputations use reported minimum periods, 69,216 and 68,097 cycles per token, workload power and SAIF durations; units are GMAC/s/mm² and pJ/useful MAC. No remaining in-scope problems found. No files changed; verdicts were not reassessed.", "summary_note": "a salutation at the start of the reviewer's summary was removed", "recomputed_rows": [ { "variant": "d015", "row": "gf180-cv_gemv_r11r_p8", "m1": 10.165055546062543, "m2": 111.02635049718413, "matches_table": true }, { "variant": "d015", "row": "gf180-vta8_official_csa", "m1": 7.008711165689114, "m2": 114.3588441492913, "matches_table": true } ], "findings": [] }, { "round": 7, "date": "2026-11-08", "revision": "r11", "vta_rows": 6, "scope": "flow sensitivity", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (sensitivity.json, sensitivity/ incl. all-vta/, review.json rounds 5-6), the comparison pages and the CV-GEMV page", "instructions_sha256": "ca0999f15cc60bafcea6dbce6422f609aa22bbb0beb2babae393af84e7e9e755", "conclusion": "confirm", "row_selection_ok": true, "summary": "The 18:48 addition holds. Independently derived omitted rows: Nangate45 vta8; ASAP7 vta8, vta8_official, vta8_tuned, vta8_official_tuned; SKY130HD and IHP SG13G2 vta8, vta8_official, vta8_official_tuned; GF180MCU vta8, vta8_official, vta8_tuned. All six tables and their folders contain exactly the required rows, with no missing or extra rows. All 64 setting.txt files show the intended variant and the other setting's base value. All 56 margin cells, best-row identities and 12 smallest-margin entries reproduce; both page tables agree within 0.05 percentage points. The 22 requested raw-file calculations reproduce within 0.1%, without collect.py; M1 units are GMAC/s/mm² and M2 units are pJ/useful MAC. Replay windows independently match SAIF header durations, and useful work matches simulated token counts. No omitted row becomes VTA's best; no metric reverses or enters ±1%. The sole added-row clock-status change is ASAP7 vta8_official_tuned at d025, missing by 14.6151 ps, disclosed on both pages. No added-row DRC or replay outcome changes. Both pages correctly report the 45/55% margins separately, remove the obsolete omitted-row limitation, and do not change verdicts or claim broader robustness. The product verdict table remains round 4. No concrete CV-GEMV-favouring design choice was found in this addition: completing all six VTA rows and selecting their best separately for each metric and variant strengthens the comparator. The narrow setting range and separate, post-result ASAP7 pair limit generalization but are disclosed. Review was read-only; different-model-family provenance could not be independently verified.", "summary_note": "a salutation at the start of the reviewer's summary was removed", "recomputed_rows": [ { "variant": "u40", "row": "gf180 vta8", "m1": 4.90863189, "m2": 133.894908244, "matches_table": true }, { "variant": "u40", "row": "gf180 vta8_official", "m1": 5.718289428, "m2": 124.71468744, "matches_table": true }, { "variant": "u40", "row": "gf180 vta8_tuned", "m1": 5.514683553, "m2": 127.364769481, "matches_table": true }, { "variant": "u50", "row": "gf180 vta8", "m1": 4.853260401, "m2": 132.290468438, "matches_table": true }, { "variant": "u50", "row": "gf180 vta8_official", "m1": 5.837634783, "m2": 122.292014584, "matches_table": true }, { "variant": "u50", "row": "gf180 vta8_tuned", "m1": 5.668642957, "m2": 125.65795054, "matches_table": true }, { "variant": "d015", "row": "gf180 vta8", "m1": 4.956249606, "m2": 130.637526673, "matches_table": true }, { "variant": "d015", "row": "gf180 vta8_official", "m1": 5.674883371, "m2": 122.847187008, "matches_table": true }, { "variant": "d015", "row": "gf180 vta8_tuned", "m1": 5.651879674, "m2": 125.675701257, "matches_table": true }, { "variant": "d025", "row": "gf180 vta8", "m1": 4.89375072, "m2": 133.357886553, "matches_table": true }, { "variant": "d025", "row": "gf180 vta8_official", "m1": 5.769136575, "m2": 122.799260072, "matches_table": true }, { "variant": "d025", "row": "gf180 vta8_tuned", "m1": 5.557085875, "m2": 126.484233915, "matches_table": true }, { "variant": "u40", "row": "asap7 vta8_official_tuned", "m1": 21911.48321151, "m2": 0.192397784, "matches_table": true }, { "variant": "u50", "row": "asap7 vta8_official_tuned", "m1": 22153.895738143, "m2": 0.19015738, "matches_table": true }, { "variant": "d015", "row": "asap7 vta8_official_tuned", "m1": 22149.218766263, "m2": 0.189983623, "matches_table": true }, { "variant": "d025", "row": "asap7 vta8_official_tuned", "m1": 21854.190625114, "m2": 0.189405163, "matches_table": true }, { "variant": "a45", "row": "asap7 vta8_official_tuned", "m1": 22017.242562768, "m2": 0.190934599, "matches_table": true }, { "variant": "a55", "row": "asap7 vta8_official_tuned", "m1": 22189.305348109, "m2": 0.189478916, "matches_table": true }, { "variant": "u40", "row": "sky130hd vta8_official_tuned", "m1": 44.543519772, "m2": 5.2882061, "matches_table": true }, { "variant": "u50", "row": "sky130hd vta8_official_tuned", "m1": 45.348494241, "m2": 5.137127497, "matches_table": true }, { "variant": "d015", "row": "sky130hd vta8_official_tuned", "m1": 44.858986624, "m2": 5.227118632, "matches_table": true }, { "variant": "d025", "row": "sky130hd vta8_official_tuned", "m1": 44.914142664, "m2": 5.155782001, "matches_table": true } ], "findings": [] }, { "round": 8, "date": "2026-11-08", "revision": "r14", "vta_rows": 6, "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/, the table before the r12 to r14 rows (before-r12/summary.json), the comparison pages and the CV-GEMV page", "instructions_sha256": "5e299ac37d506cf304cd92b68747e1859153439dcd64b9892417c34a775d3a43", "conclusion": "confirm", "conclusion_main": "confirm", "conclusion_open_pdk": "confirm", "summary": "The main code verdict is confirmed as reached; the open-PDK verdict is confirmed as not reached. Metrics use GMAC/s/mm², pJ/useful MAC and bits/useful MAC. Independent calculations matched all twelve requested CV-GEMV rows and all thirty VTA rows; the five listed VTA winners win both M1 and M2, with all six variants tied on M3. The timing rule selects r14c on Nangate45, SKY130HD and IHP, and r14r on ASAP7 and GF180MCU. Subject replay and equivalence records pass, including non-vacuous corrected IHP equivalence. GF180MCU fails G3: −0.0159037 ns setup slack, 8.02 ns minimum period, zero reported routing violations. Required rerun tables match all scalar fields: 22 main rows and 33 open-PDK rows. All 114 earlier rows remain unchanged; only thirty revision rows were added. Current-subject sensitivity calculations match, including GF180MCU density 0.25 passing setup at +0.0143414 ns without changing G3. The pages correctly retain pending-review status and list all eight GF180 revisions. Findings concern stale wording and evidence limits, not a numerical verdict reversal. Files were not modified.", "recomputed_rows": [ { "row": "nangate45-cv_gemv_r14c_p8", "m1": 627.70675146, "m2": 0.694799609, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-cv_gemv_r14r_p8", "m1": 554.31093908, "m2": 0.811972567, "m3": 5.375, "matches_summary": true }, { "row": "asap7-cv_gemv_r14c_p8", "m1": 26870.629681979, "m2": 0.137897052, "m3": 5.375, "matches_summary": true }, { "row": "asap7-cv_gemv_r14r_p8", "m1": 27686.395785111, "m2": 0.150344444, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r14c_p8", "m1": 57.251527429, "m2": 3.732905126, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r14r_p8", "m1": 53.247391343, "m2": 4.560964073, "m3": 5.375, "matches_summary": true }, { "row": "gf180-cv_gemv_r14c_p8", "m1": 7.098112032, "m2": 104.629390937, "m3": 5.375, "matches_summary": true }, { "row": "gf180-cv_gemv_r14r_p8", "m1": 10.90302979, "m2": 111.267701936, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r14c_p8", "m1": 27.69298383, "m2": 3.511850283, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r14r_p8", "m1": 26.645417822, "m2": 4.279912577, "m3": 5.375, "matches_summary": true }, { "row": "gf180-cv_gemv_r12r_p8", "m1": 10.836770204, "m2": 110.802774824, "m3": 5.375, "matches_summary": true }, { "row": "gf180-cv_gemv_r13r_p8", "m1": 10.958783236, "m2": 109.929424407, "m3": 5.375, "matches_summary": true }, { "row": "nangate45-vta8_csa", "m1": 475.216079203, "m2": 0.920621569, "m3": 6, "matches_summary": true }, { "row": "asap7-vta8_csa", "m1": 23478.483914866, "m2": 0.179309754, "m3": 6, "matches_summary": true }, { "row": "sky130hd-vta8_official_csa", "m1": 47.251586026, "m2": 4.877263251, "m3": 6, "matches_summary": true }, { "row": "gf180-vta8_official_csa", "m1": 7.204240313, "m2": 114.736359398, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2-vta8_csa", "m1": 23.304656107, "m2": 4.935926871, "m3": 6, "matches_summary": true } ], "findings": [ { "file": "site/comparison-en.html", "location": "line 1, 'Where the margin is small' final bullet", "problem": "The sensitivity summary still gives r11 margins: Nangate45 +29.9%/-23.9% and ASAP7 +18.4%/-16.0%. For r14 the smallest margins across the published setting and four variants are +32.1%/-24.2% and +17.6%/-15.4%, respectively. The later 'current subject (r14)' table is correct.", "effect": "page wording", "how_to_check": "Recompute subject/VTA-best minus one for sensitivity/latest/{u40,u50,d015,d025}/summary.json and the corresponding six VTA variant rows; compare sensitivity.json latest_subject.smallest.", "status": "fixed", "resolution": "The last bullet of 'Where the margin is small' now reads the current subject's flow-sensitivity part (r14): Nangate45 +32.1 %/-24.2 %, ASAP7 +17.6 %/-15.4 %.", "status_kr": "고침", "resolution_kr": "'여유가 작은 곳' 끝 항목이 현재 대상(r14)의 흐름 민감도 값을 읽도록 고쳤습니다: Nangate45 +32.1 %/-24.2 %, ASAP7 +17.6 %/-15.4 %." }, { "file": "corvel-comparison/RULES.md", "location": "lines 7–11; also site/comparison-en.html line 1, 'How the verdict is made'", "problem": "The introductory disclosure stops implementation changes at r11, although the current subject includes r12–r14. The dated additions and detailed page history correctly disclose those later changes.", "effect": "page wording", "how_to_check": "Compare the introductory 'r2 to r11' wording with the 06:21, 06:41 and 06:57 additions and the page's r14 history.", "status": "fixed", "resolution": "RULES.md's opening and the page's 'How the verdict is made' now say the design was improved in steps r2 to r14 (the page reads the revision from verdict.json).", "status_kr": "고침", "resolution_kr": "RULES.md 첫머리와 페이지 '판정 방식'이 r2~r14 설계 개선으로 적습니다(페이지는 verdict.json의 리비전을 읽음)." }, { "file": "corvel-comparison/REVIEW.md", "location": "lines 18, 21–31", "problem": "The brief still identifies r12 as the latest revision and says only ten r12 rows were added. There are thirty added rows covering r12–r14. It also says rounds 1 and 2 confirmed r7, whereas round 1 disagreed and round 2 confirmed after corrections.", "effect": "none", "how_to_check": "Compare REVIEW.md with RULES.md's latest additions, review.json and the before-r12/current summary row-set difference. The explicit round-8 request supersedes this stale brief.", "status": "fixed", "resolution": "REVIEW.md now names r14 as the subject, the thirty added rows of r12 to r14, the rules of 06:21, 06:41 and 06:57 and the 09:19 note, and says round 2 confirmed r7 after round 1's findings were fixed.", "status_kr": "고침", "resolution_kr": "REVIEW.md가 대상 r14, 더해진 r12~r14 30줄, 06:21·06:41·06:57 규칙과 09:19 기록을 적고, r7은 1차 지적을 고친 뒤 2차가 확인했다고 고쳐 적었습니다." }, { "file": "corvel-comparison/RULES.md", "location": "2026-11-08 06:57 addition; site/comparison-en.html line 1, VTA description", "problem": "The timing rationale supports leaving pre-register Booth recoding unapplied for the existing critical path, but does not establish that it cannot improve M1 or M2. All 20 tuned/carry-save VTA worst setup paths start after the MACs. Nevertheless, pre-register recoding and one-hot digits remain applicable, unmeasured changes; extra register bits alone do not determine net area or switching power. No missing change with a demonstrated benefit was found.", "effect": "none", "how_to_check": "Inspect rtl/vta_mvm_port_csa.v's rb registers and booth_mul, the generated CSA MAC's rB and booth_mul, and all tuned/carry-save 6_finish.rpt maximum-delay paths. Measuring alternative implementations would be required to establish their area/energy effect. The page discloses the optimization asymmetry.", "status": "kept as a limit", "resolution": "The page says why the move is not applied (every tuned and carry-save VTA row's worst path starts after its MACs, so it cannot shorten VTA's clock) and that it would add register bits; its net effect on VTA's area and energy is not measured.", "status_kr": "한계로 둠", "resolution_kr": "페이지는 적용하지 않은 이유(모든 tuned·carry-save VTA 줄의 최악 경로가 MAC 뒤에서 시작해 클록을 줄일 수 없음)와 레지스터 비트가 늘어난다는 점을 적습니다. VTA 면적·에너지에 대한 순효과는 재지 않았습니다." }, { "file": "evidence/compare/second-machine.json", "location": "rows", "problem": "This manifest names machines and hashes for only the 22 main-verdict rows, not the 33 open-PDK rows. The open-PDK second-machine summary is complete and scalar-identical, but does not itself name a machine for each row. Thus the requested per-row machine attribution is not established for the open PDKs by these files.", "effect": "none", "how_to_check": "Compare second-machine.json row keys with the 33 required keys in opdk-second-machine-summary.json. This does not change verdict.py's table-equality result; add per-row provenance to substantiate the broader attribution.", "status": "kept as stated", "resolution": "All 33 open-PDK second-machine rows ran on the Mac (none is linked from another machine); second-machine.json lists routed-netlist hashes for the 22 main rows only.", "status_kr": "그대로 둠", "resolution_kr": "오픈 PDK 두 번째 기계 33줄은 모두 맥에서 돌렸습니다(다른 기계에서 연결한 줄 없음). second-machine.json은 주 판정 22줄의 넷리스트 해시만 적습니다." }, { "file": "evidence/compare/summary.json", "location": "CV-GEMV rows, M3_basis; corresponding sim.log files", "problem": "M3 reproduces as (68096×320 + 152×112×8×12)/(896×4864) = 5.375, but the metadata schedule remains a stated accounting basis because CV-GEMV RTL and testbench are private. The logs establish the read counter value, not an independently inspectable implementation of every metadata read.", "effect": "none", "how_to_check": "Read tb_read_count in sim.log and apply M3_basis. VTA's public 64×5-bit weights and eight 8-bit activations independently give 6 bits/MAC; all six VTA rows tie on M3.", "status": "kept as stated", "resolution": "M3 for CV-GEMV is the stated accounting of its private RTL; the read counts in the replay logs match it.", "status_kr": "그대로 둠", "resolution_kr": "CV-GEMV의 M3는 비공개 RTL의 계산 방식으로 적은 값이며, 재생 기록의 읽기 수와 맞습니다." }, { "file": "site/comparison-en.html", "location": "line 1, 'Attempts': 'Each revision’s rule was committed before its result'", "problem": "The dated entries state the intended ordering, but this package does not independently prove commit-before-result chronology. The supplied rules support the revision-selection logic; they are not an immutable timestamp record of preregistration.", "effect": "none", "how_to_check": "Verify timestamped rule commits against immutable run-start/result records for r12, r13 and r14. The packaged dated prose alone cannot perform that check.", "status": "kept as stated", "resolution": "The order rule-before-result is recorded in the repository's history (each rule commit was pushed before its runs started; the VM ledger records the run times), not in this package.", "status_kr": "그대로 둠", "resolution_kr": "규칙이 결과보다 먼저라는 순서는 이 묶음이 아니라 저장소 이력(규칙 커밋을 실행 시작 전에 푸시함)과 VM 장부(실행 시각)에 남아 있습니다." } ] }, { "round": 9, "date": "2026-11-08", "revision": "r14", "vta_rows": 6, "scope": "workload breadth", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (workloads.json, workloads/, the base rows), the comparison pages", "instructions_sha256": "58a1bbd9e954bb62412bb6c6827557593476a27731eb8c637c1bbaa10b18cad3", "conclusion": "confirm", "summary": "The workload study's numerical results and both pages agree with the supplied evidence under the stated rules. Independently recomputed all 658 rows, including 603 new-workload replays and 55 base rows, without using workloads_collect.py. M1, M2 and M3 match; SAIF durations match replay windows. Every available replay reports zero mismatches and PASS. All margins, minima, empty tie/reversal lists and reported layer-12 aggregates reproduce. All 30 public W8A8 shape configs preserve the base configuration except shape parameters, build nickname and comments. Successful new-build DRC reports are empty; target misses use their own minimum periods and are disclosed. SKY130HD's unrouted build is excluded correctly. Both pages match the numbers and plainly state that no verdict changes. Findings identify provenance and measurement-scope limitations, not demonstrated metric or margin errors. M1 values above are GMAC/s/mm², M2 pJ/useful MAC, and M3 bits/useful MAC.", "recomputed_rows": [ { "row": "asap7/qwen25-l12-gate/cv_gemv_r14r_p8", "m1": 26767.85273498519, "m2": 0.13796774520208605, "m3": 5.375, "matches_summary": true }, { "row": "asap7/qwen25-l12-o/cv_gemv_r14r_p8", "m1": 26618.446479431943, "m2": 0.15362814546297035, "m3": 5.375, "matches_summary": true }, { "row": "asap7/qwen25-l12-o/vta8_csa", "m1": 23476.957131284373, "m2": 0.18140936488030882, "m3": 6, "matches_summary": true }, { "row": "asap7/qwen25-l12-q/cv_gemv_r14r_p8", "m1": 26618.446479431943, "m2": 0.1377373937166175, "m3": 5.375, "matches_summary": true }, { "row": "asap7/qwen25-l12-q/vta8_csa", "m1": 23476.957131284373, "m2": 0.16066230926906042, "m3": 6, "matches_summary": true }, { "row": "gf180/gemma4-e2b-l17-down/cv_gemv_r14r_p8", "m1": 11.078884069097912, "m2": 103.63400355563694, "m3": 5.375, "matches_summary": true }, { "row": "gf180/qwen25-l12-gate/cv_gemv_r14r_p8", "m1": 10.550585110015755, "m2": 101.54879537913389, "m3": 5.375, "matches_summary": true }, { "row": "gf180/qwen25-l12-gate/vta8_official_csa", "m1": 7.2042403128896115, "m2": 104.29776276500776, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2/qwen25-l12-gate/cv_gemv_r14c_p8", "m1": 27.028651909485266, "m2": 3.2071927536125466, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2/qwen25-l12-o/cv_gemv_r14c_p8", "m1": 26.680401515029928, "m2": 3.5454839758051655, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2/qwen25-l12-o/vta8_csa", "m1": 23.303140627415278, "m2": 4.986449429650443, "m3": 6, "matches_summary": true }, { "row": "ihp-sg13g2/qwen25-l23-down/cv_gemv_r14c_p8", "m1": 27.69298382996198, "m2": 3.3613889331788944, "m3": 5.375, "matches_summary": true }, { "row": "ihp-sg13g2/qwen25-l23-down/vta8_csa", "m1": 23.304656107139476, "m2": 4.556820307765289, "m3": 6, "matches_summary": true }, { "row": "nangate45/qwen25-l12-gate/cv_gemv_r14c_p8", "m1": 605.3289417259083, "m2": 0.6281481751589668, "m3": 5.375, "matches_summary": true }, { "row": "nangate45/qwen25-l12-gate/vta8_csa", "m1": 475.2160792031814, "m2": 0.8451266438666094, "m3": 6, "matches_summary": true }, { "row": "nangate45/qwen25-l12-gate/vta8_official", "m1": 447.00571731626104, "m2": 0.815114181336482, "m3": 6, "matches_summary": true }, { "row": "sky130hd/qwen25-l12-o/cv_gemv_r14c_p8", "m1": 54.141688072574034, "m2": 3.8697886791493943, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd/qwen25-l12-o/vta8_official_csa", "m1": 47.24851329995364, "m2": 4.919959853136023, "m3": 6, "matches_summary": true }, { "row": "sky130hd/qwen35-08b-l12-down/cv_gemv_r14c_p8", "m1": 57.038494522410666, "m2": 3.8644010759190146, "m3": 5.375, "matches_summary": true }, { "row": "sky130hd/qwen35-08b-l12-down/vta8_official_csa", "m1": 47.25145592250761, "m2": 4.858785550641223, "m3": 6, "matches_summary": true } ], "findings": [ { "file": "corvel-comparison/RULES.md", "location": "2026-11-08 08:40, 08:51 and 10:07 additions", "problem": "Evidence limitation: the supplied shape and fixture manifests match the specified projections and token positions, but the workload generator and its byte-for-byte base-reproduction log are absent. The package also does not independently establish preregistration chronology or execution-machine attribution.", "effect": "none", "how_to_check": "Supply the generator, its 15-file comparison record, and timestamped run manifests identifying machines. Fourteen published hashes alone cannot demonstrate that the generator reproduced all fifteen files.", "status": "fixed in part; the rest kept as a limit", "status_kr": "일부 고침, 나머지는 한계로 둠", "resolution": "Fixed in part: workloads.json now holds the generator's check on the base projection (base_check: 15 files compared byte for byte, 0 different; the 14 whose hashes fixtures.json publishes are listed with those hashes, all equal), and each replay row of the eleven projections names the machine it ran on. Kept as a limit: the generator itself stays out of the package (it writes CV-GEMV's input images, which come with its RTL), and the order of the rule entries and the runs is in the repository's commit times, not in the package.", "resolution_kr": "일부 고침: workloads.json에 기준 투영에서 한 생성기 검사를 넣었습니다(base_check: 파일 15개 바이트 비교, 다른 것 0, fixtures.json이 해시를 공개한 14개는 그 해시와 함께, 모두 같음). 투영 11개의 재생 줄마다 재생한 기계도 적었습니다. 한계로 둠: 생성기 자체는 CV-GEMV 입력 이미지를 쓰므로 RTL과 함께 가며 묶음에 넣지 않습니다. 규칙 항목과 실행의 순서는 묶음이 아니라 저장소의 커밋 시각에 있습니다." }, { "file": "corvel-comparison/studies/compare/power_run.py", "location": "main(): CMP_NETLISTS selection; workload result.json netlist_row fields", "problem": "Evidence limitation: every runtime-shape comparator names the expected published row, and the runner selects its routed 6_final.v. However, workload records contain row names rather than hashes of the actual netlists consumed. No substituted comparator netlist was found, but byte identity is not independently established by these records.", "effect": "none", "how_to_check": "Record the consumed 6_final.v hash for each replay and compare it with the corresponding published routed-row hash.", "status": "fixed", "status_kr": "고침", "resolution": "Each replay row of the eleven projections (603 rows; the 55 base rows are the base table's own) carries the SHA-256 of the routed netlist it read: on the build VM as it is there (hashed on the VM, vm/netlist_hashes.sh), on the AWS instance the file uploaded or built there. netlists_read shows, for each of the 55 published routed rows, that all its replays read one netlist, and that it is the netlist second-machine.json publishes for the 22 rows listed there.", "resolution_kr": "투영 11개의 재생 줄마다(603줄, 기준 55줄은 기준 표 그대로) 그 재생이 읽은 라우팅 넷리스트의 SHA-256을 적었습니다. 빌드 VM에서는 그곳의 파일을 VM에서 해시했고(vm/netlist_hashes.sh), AWS 인스턴스에서는 올린 파일이나 그곳에서 만든 파일입니다. netlists_read는 공개 라우팅 줄 55개마다 재생이 모두 넷리스트 하나를 읽었고, second-machine.json에 있는 22줄은 그 해시와 같음을 보여 줍니다." }, { "file": "corvel-comparison/RULES.md", "location": "2026-11-08 08:40: two commands and zero-time memory swap", "problem": "The zero-time swap excludes a potentially real loading cost and therefore limits the gate_proj/up_proj results to the prescribed compute-only model. Both commands' cycles and reads are included correctly. Private CV-GEMV RTL, testbench and configs prevent independent inspection of the swap implementation, shape-only rebuilds and metadata-read schedule.", "effect": "none", "how_to_check": "Sum both token-0 RESULT lines in each gate/up replay; compare the resulting cycles and reads with M1 and M3. Inspect the private implementation separately to establish behavior beyond the published accounting rule.", "status": "kept as stated", "status_kr": "그대로 둠", "resolution": "The rule (08:40) fixes the zero-time swap, as storage is outside every engine for the query between tokens too; both commands' cycles and reads are counted. The page now states the swap and that both commands are counted.", "resolution_kr": "규칙(08:40)이 시간 0 교체를 정했습니다. 토큰 사이의 질의 교체와 같이 저장소는 모든 엔진 밖에 있습니다. 두 명령의 사이클과 읽기는 모두 셉니다. 이제 페이지에 교체 방식과 두 명령을 모두 센다는 것을 적었습니다." }, { "file": "corvel-comparison/studies/compare/power_run.py", "location": "workload(): TPU/Gemmini +tok=0 and useful-MAC denominator", "problem": "TPU and Gemmini use one token while other engines use two, as allowed by the rules. Their denominators are correct. The base-workload second-token sensitivity result does not establish token sensitivity for these eleven new projections; no demonstrated advantage to CV-GEMV follows from this limitation.", "effect": "none", "how_to_check": "Replay token 1 for TPU and Gemmini on each new projection and compare per-token energy. Do not interpret eight generated token positions as eight measured replay tokens.", "status": "kept as a limit", "status_kr": "한계로 둠", "resolution": "As in the base table, the TPU and Gemmini replay one token (rules); the page now says so in this section. A second-token replay of both on the eleven projections (110 long replays) is a follow-up, not done here.", "resolution_kr": "기준 표와 같이 TPU와 Gemmini는 토큰 하나를 재생합니다(규칙). 이제 이 절에도 적었습니다. 두 엔진의 둘째 토큰 재생(투영 11개, 긴 재생 110회)은 후속으로 두고 여기서는 하지 않았습니다." }, { "file": "evidence/compare/workloads.json", "location": "by_process.sky130hd.gates.new_build_failed.cv_gemv_r14c_p8_s2432x896", "problem": "The failed build is correctly classified as G3 failure, with no subject metrics for gate/up and no subject layer-12 aggregate. Its precise reported cause—one routing violation and unrouted nets—is not independently checkable because the failing route log is absent from this package.", "effect": "none", "how_to_check": "Include the failing 5_2_route.log or a verifiable excerpt. Confirm that gate/up subject rows remain absent and that smallest.over is 10 for SKY130HD.", "status": "fixed", "status_kr": "고침", "resolution": "evidence/compare/workloads/failed-builds/sky130hd-cv_gemv_r14c_p8_s2432x896/route-excerpt.txt: the lines of its detailed-route log that report the violations left after each iteration (6,021 down to 1) and the stop on unrouted nets; the page links it.", "resolution_kr": "evidence/compare/workloads/failed-builds/sky130hd-cv_gemv_r14c_p8_s2432x896/route-excerpt.txt: 상세 라우팅 기록에서 반복마다 남은 위반 수(6,021에서 1까지)와 연결 안 된 넷으로 멈춘 줄을 담았고, 페이지에서 연결했습니다." } ] }, { "round": 10, "date": "2026-11-08", "revision": "r14", "vta_rows": 6, "scope": "workload breadth", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (workloads.json, workloads/, the base rows, review.json rounds 1 to 9), workloads.json as round 9 reviewed it, the comparison pages", "instructions_sha256": "df837a3c794b45689f45726d145afa6746d66700e3162fc1e0e742af8c0929ec", "conclusion": "disagree", "summary": "No unauthorized JSON differences were found: every existing field, including metrics, margins, aggregates and gates, is unchanged. All 603 non-base hashes are present. The 55 published-row hash groups reproduce exactly; each uses one hash across machines, and all 22 second-machine references match. The 59 new-build hashes are individually consistent, unique and distinct from published-row hashes. All 14 base hashes match fixtures.json; 15 comparisons, zero differences and one unpublished hash are disclosed. The route excerpt supports the failure description and exclusion. Both pages preserve all five findings verbatim and correctly limit round 9 to workload breadth. All 110 TPU/Gemmini logs contain token-0 replay only, consistent with the retained limit. Disagreement concerns two reporting overstatements, not numerical changes.", "recomputed_rows": [], "findings": [ { "file": "evidence/compare/review.json", "location": "rounds[round=9].findings[0].status; both pages, #review, round 9 finding 1", "problem": "The status 'fixed' overstates closure. The resolution explicitly retains the absent generator and unavailable preregistration chronology evidence. Added base-check results and machine labels address parts of the original finding, but do not resolve all of it. Label this partially addressed with remaining limits, or split its dispositions.", "effect": "page wording", "how_to_check": "Compare finding 1's original problem with its resolution: the latter explicitly says the generator remains outside the package and chronology resides in repository commit times outside it.", "status": "fixed", "status_kr": "고침", "resolution": "Round 9 finding 1 is now marked 'fixed in part; the rest kept as a limit', and its resolution says which part is fixed (the base check, the machines) and which is kept as a limit (the generator, the order of the runs).", "resolution_kr": "9차 지적 1의 상태를 '일부 고침, 나머지는 한계로 둠'으로 바꾸고, 고친 부분(기준 재현 기록, 기계)과 한계로 둔 부분(생성기, 실행 순서)을 처리 내용에 나눠 적었습니다." }, { "file": "evidence/compare/review.json; site/comparison-en.html; site/comparison-kr.html", "location": "round 9 finding 2 resolution; both pages #workloads provenance paragraph and #review finding 2", "problem": "The wording says every row carries its replay machine and netlist hash. Only the 603 non-base rows do; all 55 base rows lack these additions. Qualify this as 'every new-workload replay row'. The requested non-base coverage itself passes.", "effect": "page wording", "how_to_check": "Partition workloads.json.rows by workload == 'base' and inspect machine/netlist_sha256 presence. The page's 311 AWS plus 292 Azure count totals 603, not 658.", "status": "fixed", "status_kr": "고침", "resolution": "Both pages and round 9's resolution now say that each replay row of the eleven projections (603) names its machine and netlist hash, and that the 55 base rows are the base table's own.", "resolution_kr": "두 페이지와 9차 처리 내용에 투영 11개의 재생 줄(603줄)마다 기계와 넷리스트 해시를 적었고, 기준 55줄은 기준 표 그대로라고 밝혔습니다." } ] }, { "round": 11, "date": "2026-11-08", "revision": "r14", "vta_rows": 6, "scope": "workload breadth", "reviewer": "OpenAI Codex CLI 0.166.0, model gpt-6-astra, reasoning effort low, read-only sandbox", "material": "public material only: reproduce.zip, public/evidence/compare/ (workloads.json, review.json rounds 1 to 10), workloads.json and review.json as round 10 reviewed them, the comparison pages", "instructions_sha256": "041d55768769d385b1f4170b824cc9468b55bd9b4821bb8d691b18e6a16cf48c", "conclusion": "confirm", "summary": "Both round 10 wording findings are resolved in English and Korean. Round 9 finding 1 distinguishes fixed base-check and machine attribution from retained generator and chronology limits. Provenance wording correctly separates 603 replay rows across eleven projections (311 AWS, 292 Azure), all carrying machine and netlist hashes, from 55 base rows. workloads.json is byte-identical to its snapshot. Existing review.json changes are limited to six permitted status/resolution fields; round 10 is added. The top-level conclusion remains round 8's confirm. Both pages preserve rounds 9 and 10's findings verbatim, their confirm/disagree conclusions, and workload-only scope. No files were changed.", "recomputed_rows": [], "findings": [] }, { "round": 12, "date": "2026-11-10", "revision": "r14", "vta_rows": 6, "scope": "256 MACs per cycle", "reviewer": "Antigravity CLI 1.5.3, model Gemini 4 Argon, effort low, terminal sandbox, run in a copy of the public files (OpenAI Codex, used in rounds 4 to 11, was at its usage limit)", "material": "public material only: reproduce.zip, public/evidence/compare/ (size256.json, size256/, workloads256.json, workloads256/, bandwidth-sizing.json, the base rows), the comparison pages", "instructions_sha256": "c10a39b64d58a7f24d5e577a5a823f44fbc8a0de14d0dd42fe6893c17ef14f5d", "conclusion": "confirm", "summary": "The 256-MAC study and its reporting on the English and Korean pages stand on the raw files under the rules. All 55 base builds and 11 workload projections match the 16:51 and 17:49 entries; all 50 public 256-MAC comparator config.mk files differ from their 64-MAC counterparts only in the engine size, and the two fallback rows (asap7-cv_gemv_r14r_p32 and gf180-cv_gemv_r14r_p32) match first_attempts and are disclosed on the pages. Independent recomputations of M1, M2, and M3 from 6_report.json, 6_finish.rpt, sim.log, power.log, result.json, and activity-header.txt match size256.json and workloads256.json across all five processes. Every margin, 64-MAC ratio, reversal/near-tie list, layer-12 M2 sum, and bandwidth-sizing table entry (including LPDDR5X-8533 x64 and DDR4-3200 x64 recomputations) is exact. Every replay passes Gate G1 (0 mismatches, PASS) and every build has 0 DRC violations, with setup misses scored at 1/period_min. Minor non-metric issues were noted in machine-name heuristics in workloads_collect.py and size_collect.py and in the HTML prose omitting the 14 new-build setup misses recorded in workloads256.json.", "recomputed_rows": [ { "row": "size256/nangate45-cv_gemv_r14c_p32", "m1": 649.0708570364492, "m2": 0.7040676018690525, "m3": 4.625, "matches_summary": true }, { "row": "size256/nangate45-vta16_official", "m1": 577.8650318956902, "m2": 0.6764021082354553, "m3": 5.5, "matches_summary": true }, { "row": "size256/asap7-cv_gemv_r14r_p32", "m1": 28027.808308043248, "m2": 0.1542395017397314, "m3": 4.625, "matches_summary": true }, { "row": "size256/asap7-vta16_official", "m1": 30416.247423293138, "m2": 0.12245891592603411, "m3": 5.5, "matches_summary": true }, { "row": "size256/sky130hd-cv_gemv_r14c_p32", "m1": 54.90341136544639, "m2": 4.2992118285215, "m3": 4.625, "matches_summary": true }, { "row": "size256/sky130hd-vta16_official_csa", "m1": 54.409781431843584, "m2": 3.8085090042144083, "m3": 5.5, "matches_summary": true }, { "row": "size256/gf180-cv_gemv_r14r_p32", "m1": 10.651258015061956, "m2": 111.54224270148026, "m3": 4.625, "matches_summary": true }, { "row": "size256/gf180-vta16_official", "m1": 8.057359889775906, "m2": 88.76221429297884, "m3": 5.5, "matches_summary": true }, { "row": "size256/ihp-sg13g2-cv_gemv_r14c_p32", "m1": 29.211784745184428, "m2": 3.4982834742495887, "m3": 4.625, "matches_summary": true }, { "row": "size256/ihp-sg13g2-vta16_csa", "m1": 27.662805519561815, "m2": 3.5648915234766214, "m3": 5.5, "matches_summary": true }, { "row": "size256/ihp-sg13g2-vta16_official_csa", "m1": 27.527869648386293, "m2": 3.564057013663844, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/nangate45-cv_gemv_r14c_p32_s2432x896", "m1": 626.9524313749703, "m2": 0.6250459423874016, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/nangate45-vta16_official", "m1": 577.8650318956902, "m2": 0.5764940897015564, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l23-down/nangate45-cv_gemv_r14c_p32", "m1": 649.0708570364492, "m2": 0.6632372942656324, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l23-down/nangate45-vta16_official", "m1": 577.8650318956902, "m2": 0.5946174869019587, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/nangate45-cv_gemv_r14c_p32_s896x896", "m1": 630.8884609972396, "m2": 0.7002337632284359, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/nangate45-vta16_official", "m1": 577.7147557687439, "m2": 0.6894402614391093, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/asap7-cv_gemv_r14r_p32_s896x896", "m1": 26765.44439982085, "m2": 0.15758238520408166, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/asap7-vta16_official", "m1": 30408.337555751892, "m2": 0.12440298210741939, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/asap7-vta16_official_csa", "m1": 29812.640196966375, "m2": 0.12421426177318338, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-q/asap7-cv_gemv_r14r_p32_s896x896", "m1": 26765.44439982085, "m2": 0.1408439918237803, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-q/asap7-vta16_official", "m1": 30408.337555751892, "m2": 0.10553192610423807, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen35-08b-l12-down/asap7-cv_gemv_r14r_p32_s1024x3584", "m1": 28326.94537102842, "m2": 0.15248364255496433, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen35-08b-l12-down/asap7-vta16_official", "m1": 30415.912446283823, "m2": 0.12144079416189194, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/sky130hd-cv_gemv_r14c_p32_s896x896", "m1": 53.62135982722761, "m2": 4.318937908063616, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-o/sky130hd-vta16_official_csa", "m1": 54.39563235020969, "m2": 3.838373109905009, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l23-down/sky130hd-cv_gemv_r14c_p32", "m1": 54.90341136544639, "m2": 4.0350934275691674, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l23-down/sky130hd-vta16_official", "m1": 51.859472692522566, "m2": 3.5672294011543673, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-k/sky130hd-cv_gemv_r14c_p32_s128x896", "m1": 55.02477229367099, "m2": 3.6413147216796875, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-k/sky130hd-vta16_official_csa", "m1": 54.291904665242775, "m2": 3.4024189022827147, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/gf180-cv_gemv_r14r_p32_s2432x896", "m1": 10.594456468186252, "m2": 102.27746039139595, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/gf180-vta16_official", "m1": 8.057359889775906, "m2": 76.03572086276864, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen35-08b-l12-down/gf180-cv_gemv_r14r_p32_s1024x3584", "m1": 11.330992136500855, "m2": 110.10321206120082, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen35-08b-l12-down/gf180-vta16_official", "m1": 8.057271126569258, "m2": 87.92689840469497, "m3": 5.5, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/ihp-sg13g2-cv_gemv_r14c_p32_s2432x896", "m1": 28.111270808867605, "m2": 3.175401759262659, "m3": 4.625, "matches_summary": true }, { "row": "workloads256/qwen25-l12-gate/ihp-sg13g2-vta16_csa", "m1": 27.662805519561815, "m2": 3.202432969288417, "m3": 5.5, "matches_summary": true } ], "findings": [ { "effect": "none", "file": "corvel-comparison/studies/compare/workloads_collect.py", "how_to_check": "Compare workloads_collect.py lines 174-205 with evidence/compare/workloads256.json lines 18947-19443 and evidence/compare/workloads256//-/netlist-sha256.txt.", "location": "lines 174-205 (and evidence/compare/workloads256.json netlists_read lines 18947-19443)", "problem": "machine(run) in workloads_collect.py checks only whether 'aws' or 'pnr' is in the run folder name, so the 515 non-base replays that reused the 55 base netlists (in run folder ext4b-256-replay) are misclassified as 'Azure build VM', and netlist_sha() then looks in vm_netlists() (64-MAC netlists only) instead of reading netlist-sha256.txt or size256.json. As a result, workloads256.json netlists_read records sha256_by_machine={'Azure build VM': [null]}, one_netlist=false, and equals_size256_json=false for all 55 base builds (and appends null to the 5 w8a8_p32 entries), even though the actual netlist-sha256.txt files in evidence/compare/workloads256/ match size256.json.", "status": "fixed", "status_kr": "고침", "resolution": "workloads_collect.py now records the AWS instances for every 256-MAC projection row (17:49, part 2: all runs on AWS), whatever the run folder's name, and hashes each replayed row's routed netlist as uploaded from the 256-MAC study's own folder. netlists_read now shows, for all 55 rows, that every replay read one netlist and that it equals the one size256.json publishes. No metric changed. The replays ran in the folders ext3wl-replay, ext3wl-replay2 and ext3wl-replay3; the folder named in the finding is not one of them.", "resolution_kr": "workloads_collect.py가 이제 256 MAC 투영 줄을 폴더 이름과 상관없이 모두 AWS 인스턴스로 적습니다(17:49 2부: 모두 AWS에서 실행). 재생한 줄의 라우팅 넷리스트는 256 MAC 측정 폴더에서 올린 그 파일로 해시합니다. 이제 netlists_read는 55줄 모두 재생이 넷리스트 하나를 읽었고 그것이 size256.json의 넷리스트와 같음을 보여 줍니다. 바뀐 지표는 없습니다. 재생은 ext3wl-replay, ext3wl-replay2, ext3wl-replay3 폴더에서 돌았고, 지적에 적힌 폴더 이름은 그중에 없습니다." }, { "effect": "none", "file": "corvel-comparison/studies/compare/size_collect.py", "how_to_check": "Compare size_collect.py line 70 and evidence/compare/size256.json lines 2381-2420 with corvel-comparison/RULES.md entry 2026-11-10 08:55 and site/comparison-en.html line 788.", "location": "line 70 (and evidence/compare/size256.json reproduction.machines line 2383)", "problem": "machine(run) in size_collect.py labels any run folder without 'aws' in its name as 'Azure build VM'. Because four of the five reproduction folders (ext3rep-azure, ext3rep-azure-b, ext3rep-hold-asap7, ext3rep-hold-ng45) do not contain 'aws' in their names even though RULES.md 2026-11-10 08:55 and the HTML pages state all 55 reproduction builds ran on AWS instances, size256.json reproduction.machines lists ['AWS instance', 'Azure build VM']. In addition, only the aggregate reproduction summary in size256.json is published, not the second run's per-row raw files.", "status": "fixed", "status_kr": "고침", "resolution": "size_collect.py now names the machine by folder: only the Azure build VM's first-run folder is the Azure VM; the ASAP7 fallback builds and every reproduction folder ran on AWS instances. This also corrected the first-run machine counts on both pages (AWS instance 23 rows, Azure build VM 32; they read 17 and 38 because the ASAP7 fallback folder's name has no 'aws'). size256.json's reproduction now lists, per row, the second run's folder, machine and its values of the nine compared fields, and the pages say where. No metric changed.", "resolution_kr": "size_collect.py가 이제 이름이 아니라 폴더로 기계를 정합니다. Azure 빌드 VM의 첫 실행 폴더만 Azure이고, ASAP7 대체 설정 빌드와 모든 재현 폴더는 AWS 인스턴스에서 돌았습니다. 이로써 두 페이지의 첫 실행 기계 수도 바로잡혔습니다(AWS 인스턴스 23줄, Azure 빌드 VM 32줄. ASAP7 대체 설정 폴더 이름에 'aws'가 없어 17·38줄로 나왔음). size256.json의 reproduction에는 줄마다 두 번째 실행의 폴더, 기계, 비교한 9개 항목 값을 넣었고, 페이지에 그 위치를 적었습니다. 바뀐 지표는 없습니다." }, { "effect": "page wording", "file": "site/comparison-en.html", "how_to_check": "Compare site/comparison-en.html lines 791-817 and site/comparison-kr.html lines 774-800 against evidence/compare/workloads256.json by_process..gates.new_build_g3_fail and corvel-comparison/RULES.md entries 2026-11-08 08:40 and 17:49 part 2.", "location": "line 798 (and site/comparison-kr.html line 781)", "problem": "RULES.md 17:49 part 2 requires the eleven 256-MAC projections to be reported as in the 08:40 entry ('Gate G1 (zero mismatches) is required on every replay; Gate G3 is checked on every new build and reported'). In workloads256.json, 14 of the 60 new shape builds missed the 8.0 ns target clock (sky130hd cv_gemv_r14c_p32_s896x896 at -0.030 ns, all 12 new gf180 builds at -0.115 to -1.086 ns, and ihp-sg13g2 w8a8_p32_s1536x12288 at -0.007 ns) and were scored at their own minimum periods, but the 'Other layers and models at 256 MACs' section on both HTML pages mentions only '605 replays, every output bit-exact' and omits the 14 Gate G3 setup misses on the 60 new builds.", "status": "fixed", "status_kr": "고침", "resolution": "Both pages' section on the eleven projections at 256 MACs now lists the 14 new builds that miss the target clock, each with its setup miss, says they are scored at their own minimum period, and states route DRC 0 on every new build.", "resolution_kr": "두 페이지의 256 MAC 투영 절에 목표 클록을 맞추지 못한 새 빌드 14개와 각각의 미달 폭을 적고, 자기 최소 주기로 계산했으며 새 빌드의 라우팅 DRC는 모두 0이라고 적었습니다." }, { "effect": "none", "file": "corvel-comparison/studies/compare/workloads_collect.py", "how_to_check": "Compare workloads_collect.py lines 97-102 with evidence/compare/workloads256.json by_process..smallest.M2 and by_process..workloads..margins_pct.M2.", "location": "lines 97-102 (and evidence/compare/workloads256.json by_process.*.smallest.M2)", "problem": "Because M2 margins are stored as (subject - best) / best (positive when CV-GEMV has higher energy per MAC and loses, negative when CV-GEMV has lower energy and wins) and smallest_margin() minimizes -kv[1] for M2, workloads256.json by_process.*.smallest.M2 selects the most positive M2 margin (CV-GEMV's worst M2 loss on Nangate45, ASAP7, SKY130HD, and GF180MCU; smallest M2 win on IHP SG13G2) rather than the cell closest to 0% in magnitude. The HTML pages state the full M2 margin ranges accurately.", "status": "kept as stated", "status_kr": "그대로 둠", "resolution": "By the 08:40 entry, as at 64 MACs, the smallest margin is the cell least favourable to CV-GEMV: for M2 (lower is better) the largest (subject - best) / best. The pages now say so next to the table.", "resolution_kr": "08:40 항목에 따라 64 MAC과 같이, 가장 작은 우위는 CV-GEMV에 가장 불리한 칸입니다. M2(낮을수록 좋음)에서는 (대상 - 최고) / 최고가 가장 큰 칸입니다. 이제 페이지의 표 앞에 그렇게 적었습니다." } ], "summary_note": "Two statements in the reviewer's summary are not supported by the files: the sizing table has no DDR4-3200 x64 memory (its memories are LPDDR4X, LPDDR5, LPDDR5X and one HBM3 stack), and the rows built with a fallback are seven VTA rows (the six ASAP7 VTA rows and Nangate45 vta16_official, hold repair off after the flow stopped at the hold-repair buffer limit), not CV-GEMV rows; first_attempts and the pages list those seven." }, { "round": 13, "date": "2026-11-10", "revision": "r14", "vta_rows": 6, "scope": "256 MACs per cycle", "reviewer": "Antigravity CLI 1.5.3, model Gemini 4 Argon, effort low, terminal sandbox, run in a copy of the public files (OpenAI Codex, used in rounds 4 to 11, was at its usage limit)", "material": "public material only: reproduce.zip, public/evidence/compare/ (size256.json, size256/, workloads256.json, workloads256/, bandwidth-sizing.json, the base rows), the comparison pages", "instructions_sha256": "e9c84fe1008e9035f95761c3a0bff7015b22a9eeef71ba8c5ea99e8238c305b4", "conclusion": "confirm", "summary": "All four Round 12 findings are resolved and nothing else changed. In workloads256.json, all 605 non-base rows name \"AWS instance\" as machine with a 64-hex netlist_sha256, and netlists_read shows one netlist matching size256.json for all 55 rows, matching RULES.md. size_collect.py names machines by folder; size256.json has 23 \"AWS instance\" and 32 \"Azure build VM\" first-run rows matching both HTML pages, and reproduction.rows has 55 entries whose nine compared fields match all 55 first-run rows, with both pages citing size256.json (reproduction, rows). Both pages list all 14 new builds (21 rows in workloads256.json) missing target clock with their setup misses and state minimum-period scoring and 0 route DRC across all 60 new builds (90 rows). Both pages define smallest margin as the cell least favourable to CV-GEMV, matching workloads_collect.py. Diffs against evidence/before-r13/ show no other changes, unchanged M1/M2/M3, verified review.json round 12 summary_note, and round 8's \"confirm\" conclusion.", "recomputed_rows": [], "findings": [] }, { "round": 14, "date": "2026-11-10", "revision": "r15 (not the verdict subject, which stays r14)", "vta_rows": 6, "scope": "16 x 16 at 256 MACs", "reviewer": "Antigravity CLI 1.5.4, model Gemini 4 Argon, effort low, terminal sandbox, run in a copy of the public files (OpenAI Codex, used in rounds 4 to 11, was at its usage limit)", "material": "public material only: reproduce.zip, public/evidence/compare/ (lanes256.json, lanes256/, size256.json, size256/, the base rows), the comparison pages", "instructions_sha256": "cac3bb54bdbeae704c9b95f1dba8715ec57bea77b70c15d14f30aa2ef0090be5", "conclusion": "confirm", "summary": "Confirmed Round 14 (CV-GEMV r15 at 16x16, 256 MACs/cycle) from the raw files. (1) Applying the 2026-11-10 11:20 timing rule to 6_report.json, 6_finish.rpt, and 5_route_drc.rpt selects PAIR_REG=0 on Nangate45 and IHP SG13G2, PAIR_REG=2 on ASAP7 and SKY130HD, and PAIR_REG=1 on GF180MCU (shortest minimum period, 8.99 ns), matching lanes256.json and both HTML pages; lanes_collect.py makes this choice from flow reports before opening replay or power files. (2) Recomputed M1, M2, and M3 across all 15 builds match lanes256.json to full precision, with result.json windows matching SAIF DURATION in activity-header.txt. M3 (4.875 bits/MAC) uses collect.py's CV-GEMV formula with 16x16 replacing 8x8, and 16x16x4 + 16x8 = 1152 bits/read is exact for 16 rows of 16 INT4 weights plus 16 INT8 activations. (3) All comparator choices, margins, r14 margins, over_r14 ratios, and reversal lists (Nangate45 M2, ASAP7 M1, SKY130HD M1/M2 turned; ASAP7 M2 +4.3% and GF180MCU M2 +21.3% remain reversed) are exact. (4) All 15 builds pass G1 and have 0 route DRC violations; all 5 chosen rows reproduce identically across 9 fields on the other AWS instance. (5) Both HTML pages match lanes256.json and state all disclosures.", "recomputed_rows": [ { "row": "asap7-cv_gemv_r15c_p16t16", "m1": 29567.902953141136, "m2": 0.12736627450813925, "m3": 4.875, "matches_summary": true }, { "row": "asap7-cv_gemv_r15o_p16t16", "m1": 33260.869223308786, "m2": 0.1277114793854907, "m3": 4.875, "matches_summary": true }, { "row": "asap7-cv_gemv_r15r_p16t16", "m1": 32355.152074037418, "m2": 0.13784711632979546, "m3": 4.875, "matches_summary": true }, { "row": "gf180-cv_gemv_r15c_p16t16", "m1": 7.451947930079859, "m2": 96.11701523217224, "m3": 4.875, "matches_summary": true }, { "row": "gf180-cv_gemv_r15o_p16t16", "m1": 9.223449350421774, "m2": 102.03311305454798, "m3": 4.875, "matches_summary": true }, { "row": "gf180-cv_gemv_r15r_p16t16", "m1": 10.963520112445506, "m2": 107.70813630389449, "m3": 4.875, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r15c_p16t16", "m1": 31.918430560794814, "m2": 3.099054034836848, "m3": 4.875, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r15o_p16t16", "m1": 30.68998196711175, "m2": 3.311368479793233, "m3": 4.875, "matches_summary": true }, { "row": "ihp-sg13g2-cv_gemv_r15r_p16t16", "m1": 30.138794601884122, "m2": 3.8829798629875474, "m3": 4.875, "matches_summary": true }, { "row": "nangate45-cv_gemv_r15c_p16t16", "m1": 733.2945607352068, "m2": 0.6479744477511528, "m3": 4.875, "matches_summary": true }, { "row": "nangate45-cv_gemv_r15o_p16t16", "m1": 692.8590166370492, "m2": 0.6876691836203668, "m3": 4.875, "matches_summary": true }, { "row": "nangate45-cv_gemv_r15r_p16t16", "m1": 645.9098027329846, "m2": 0.7601231718272733, "m3": 4.875, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r15c_p16t16", "m1": 58.69033075748893, "m2": 4.0019915653085345, "m3": 4.875, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r15o_p16t16", "m1": 64.25938130158751, "m2": 3.708494044804394, "m3": 4.875, "matches_summary": true }, { "row": "sky130hd-cv_gemv_r15r_p16t16", "m1": 60.813816899199495, "m2": 4.2549570932947605, "m3": 4.875, "matches_summary": true } ], "findings": [ { "effect": "none", "file": "corvel-comparison/RULES.md", "how_to_check": "Read corvel-comparison/RULES.md lines 331-383, evidence/compare/lanes256.json (checks_before_flow, by_process), and site/comparison-en.html / site/comparison-kr.html (#lanes256 and #limits); verify that no files exist for CV-GEMV RTL, testbench, or flow configs.", "location": "lines 331-383 (2026-11-10 11:20 and 19:35); site/comparison-en.html#lanes256", "problem": "Methodological and evidence limits that could favour CV-GEMV r15 in this study: (1) the 16x16 shape and the PAIR_REG=2 half-way dot-product pipeline option were fixed after inspecting 256-MAC r14 results and synthesis areas (before any r15 place-and-route flow, replay, or power run); (2) three builds per process (PAIR_REG = 0, 2, 1) are built and one is selected by the timing-only rule while the 256-MAC comparator rows from size256.json are reused without rebuilding; and (3) CV-GEMV r15 RTL, testbench, and flow configs remain private, so M3 metadata-read accounting and identical flow settings (beyond constraint.sdc and report headers) cannot be inspected from source. All three points are explicitly disclosed in RULES.md and on both HTML pages, all 15 builds are reported with their full metrics, and the verdicts' subject remains r14.", "status": "fixed", "resolution": "RULES.md states all three points (the 11:20 entry). The pages stated the second and third (three builds with one reported by the timing rule and all fifteen shown; the comparator rows not rebuilt; r15's RTL not public) but not the first. Both pages now say in the section's first paragraph that the shape and the PAIR_REG = 2 setting were fixed after the 256-MAC results, from RTL replays and synthesis areas, before any flow of an r15 row, and that a second shape (8 rows of 32 columns) was looked at by synthesis only and is not measured. No number changed.", "status_kr": "고침", "resolution_kr": "RULES.md(11:20 항목)에는 세 가지가 모두 적혀 있습니다. 페이지에는 둘째와 셋째(빌드 세 개 중 타이밍 규칙으로 하나를 보고하고 15개를 모두 실음, 비교 줄은 다시 빌드하지 않음, r15의 RTL 비공개)만 있었고 첫째가 없었습니다. 두 페이지 구역 첫 문단에 모양과 PAIR_REG = 2 설정을 256 MAC 결과를 본 뒤 r15 줄의 흐름이 하나도 없을 때 RTL 재생과 합성 면적으로 정했다는 것, 다른 모양 하나(8행 × 32열)는 합성으로만 살펴보고 재지 않았다는 것을 적었습니다. 숫자는 바뀌지 않았습니다." } ], "summary_note": "Two details of the reviewer's text differ from the files: the finding's how-to-check names a field checks_before_flow of lanes256.json, which is the field checks; and when the reviewer read them the pages stated two of the finding's three points, not that the shape and the PAIR_REG = 2 setting were fixed after the 256-MAC results (RULES.md stated it). That sentence was added to both pages after this round." } ], "conclusion": "confirm" }