Step 3.7 Flash / apple-product-landing

Model evaluation sequence vision-description-v1

Historical v2 reprojection of vision-description-v1; generation timing was not recorded.

2026-07-12T00:41:27.469Z
20 tasks

Model Setup

Step 3.7 Flash

Model Setup identity
sha256:f97b938f62fc4369a575b5092ef78c5de13c222bbd1c7dd22e8e35b53a3010e3

Base model
Step 3.7 Flash

Quantization
Q3_K_L

Results

Step 3.7 Flash / apple-product-landing

visual-web-animation

manual_review

Wall time: 70.2 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:4af09be7e8c084d3399c3071658d0cd684c7ecb393f925cc93ccaa5a9d9b3b44
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / stripe-homepage

visual-web-animation

reject

Wall time: 72.9 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:28323ab04f22b73df958ad40318ef04d6f909f4fb75957c9d0bb3085816446c9
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

No publishable generated artifact for this task.

Step 3.7 Flash / linear-homepage

visual-web-animation

reject

Wall time: 72.3 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:c012672acab625d94d5b4e70d2d275820b9751c9c536d44dd71cebc775461825
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

No publishable generated artifact for this task.

Step 3.7 Flash / vercel-homepage

visual-web-animation

manual_review

Wall time: 61.2 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:ede5b85cc44f43f94c3a6b7493d4e0de1cba11c23277f345b8a69798a3c8ff8f
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / notion-homepage

visual-web-animation

reject

Wall time: 83.8 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:5e8549395fd923644bfd75fd66425904b7aa7d1eff3325de5a372a31ae7da629
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

No publishable generated artifact for this task.

Step 3.7 Flash / github-repository

visual-web-animation

reject

Wall time: 95.5 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:6cfc118eab6501cf7fb308d7b786a6a91fe4216c41a08e4b1fc63e19dddd0178
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

No publishable generated artifact for this task.

Step 3.7 Flash / github-issues

visual-web-animation

manual_review

Wall time: 157.2 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:0213865ef9b448cfc6119c3a12ddf0ee3ced14b8061e1026d884096a5c1f9cc5
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / wikipedia-article

visual-web-animation

manual_review

Wall time: 68.7 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:194201c494da465509dce67932c28f0145d22a4c83c2fe69f33dfe5ce7702772
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / ap-news-article

visual-web-animation

reject

Wall time: 63.7 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:48dbae2fbaca8b61747b99897e1430b31f1e1656f9d308cda164e0ac7b5178aa
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

No publishable generated artifact for this task.

Step 3.7 Flash / youtube-watch

visual-web-animation

manual_review

Wall time: 72.8 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:30132c1605a2c688154e428f12f3f5bfb412dd8ac4d5ed0e775fa6235e5cebf8
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / public-discussion-thread

visual-web-animation

manual_review

Wall time: 58.3 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:4659ba2231db71f3c23418f6fcf400022a6632949d03f98040818f225fa66e7e
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / x-profile

visual-web-animation

manual_review

Wall time: 76.2 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:43a6efd9a0037ca7ed82f2047848a157f3a16cd5db14385d49e43afd080b1e5f
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / ikea-product-page

visual-web-animation

manual_review

Wall time: 102.7 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:9cf6969253c6db4209cda17fa3dd90d7ac28f8cb912bb93812a492261145f0d7
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / airbnb-search

visual-web-animation

manual_review

Wall time: 82.3 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:79746a11e83423408597c1e19d3e6aa0ea51a97aaf94a021f85ee9ad28495dee
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / craigslist-apartments

visual-web-animation

manual_review

Wall time: 105.7 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:8947da556f924e12fc69696c8658aab938803b4adb8fa21211b84b0a06fee965
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / target-search

visual-web-animation

manual_review

Wall time: 90.6 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:aa6ff17297c0f8c3942d3d762e545f53203be60c7cf096697fefeffef6f6464f
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / spotify-album

visual-web-animation

manual_review

Wall time: 100.3 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:3052231d8546cbb1ef4c0977c380b5614b17430010016bee30266e131800fb3a
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / hn-algolia-search-results

visual-web-animation

manual_review

Wall time: 75.4 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:a756b68eb48a2d9a395bcdb3b05efcbf7bcc995729aaf02956368f4d1f0fb093
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / espn-scoreboard

visual-web-animation

manual_review

Wall time: 86.1 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:12b1eb6e841f9c5d0f26902b3bdd72b2f6e991543de1b5070270aac111beb25d
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Step 3.7 Flash / hacker-news

visual-web-animation

manual_review

Wall time: 64.9 s
Generation time: unknown
Generation TPS: unknown
Decoded tokens: unknown
Timing source: unknown
Output tokens: unknown

Attempt Fidelity

Task protocol: sha256:ec60cb6e8559f701f7e6fa8ab00302064530de827c6ad22d9cb61fbda7b26282
Assistance: native
Comparison: Eligible for automatic comparison consideration
Comparison reasons: none

Stewart Notes

# Stewart Notes

Model evaluation sequence vision-description-v1

- Add observations here.

Edit notes.md in this static run folder to add observations without rerunning the model.