From 4028c587b1853656e86cfb77d6075fb2a4c12dcc Mon Sep 17 00:00:00 2001 From: Gahow Wang Date: Mon, 25 May 2026 11:41:31 +0800 Subject: [PATCH] Paper section: system analysis + workload figures + KV-wall model Adds the system-level argument resolving the roofline/PD-sep paradox. Even at 95% cache reuse prefill stays compute-bound (the C6 roofline fact), yet PD separation regresses TTFT 72%. The new system_analysis.md walks through six layers showing why the roofline claim is necessary but not sufficient, with the falsifiable condition being decode-side KV memory budget: concurrent_decode * KV_per_req / (N_D * HBM_pool). For chatbot this ratio is << 1 at any layout; for agentic at p90+ context it goes >> 1 under 4P+4D and 6P+2D, predicting the empirical 97% decode KV occupancy. fig_kv_memory_wall.pdf visualizes the model with audit-able constants; fig_c1a/b ground the per-request KV-size inputs in the actual sampled trace (input p50=33.5k, p90=101k, intra-session reuse 79.2%). Co-Authored-By: Claude Opus 4.7 --- analysis/pd_sep_paper_section/README.md | 37 ++- .../figures/fig_c1a_io_cdf.pdf | Bin 0 -> 22776 bytes .../figures/fig_c1b_reuse.pdf | Bin 0 -> 19643 bytes .../figures/fig_kv_memory_wall.pdf | Bin 0 -> 23568 bytes .../scripts/plot_kv_memory_wall.py | 127 ++++++++++ .../scripts/plot_workload.py | 34 ++- .../pd_sep_paper_section/system_analysis.md | 226 ++++++++++++++++++ 7 files changed, 404 insertions(+), 20 deletions(-) create mode 100644 analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf create mode 100644 analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf create mode 100644 analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf create mode 100644 analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py create mode 100644 analysis/pd_sep_paper_section/system_analysis.md diff --git a/analysis/pd_sep_paper_section/README.md b/analysis/pd_sep_paper_section/README.md index aeb7e45..0fd3fa5 100644 --- a/analysis/pd_sep_paper_section/README.md +++ b/analysis/pd_sep_paper_section/README.md @@ -9,28 +9,46 @@ not the whole paper. ``` analysis/pd_sep_paper_section/ ├── README.md # this file +├── system_analysis.md # why PD-sep loses despite compute-bound prefill (6 layers) ├── scripts/ │ ├── plot_workload.py # C1: input/output CDF + KV reuse decomposition │ ├── plot_roofline.py # C6: prefill roofline at varying cache reuse -│ └── plot_routing_lever.py # C7: routing vs PD-sep as design levers +│ ├── plot_routing_lever.py # C7: routing vs PD-sep as design levers +│ └── plot_kv_memory_wall.py # KV mem-wall: the system-level explanation └── figures/ - ├── fig_c6_roofline.pdf # rendered locally (analytical, no trace needed) - ├── fig_c7_routing_lever.pdf # rendered locally (from REPORT.md §3.1) - └── (fig_c1a_io_cdf.pdf, # produced on dash0 when trace is available - fig_c1b_reuse.pdf) + ├── fig_c1a_io_cdf.pdf # input/output token CDF (from traces/w600_r0.0015_st30.jsonl) + ├── fig_c1b_reuse.pdf # KV reuse decomposition: 79% intra-session + ├── fig_c6_roofline.pdf # analytical roofline + ├── fig_c7_routing_lever.pdf # routing vs PD-sep (legacy data, footer caveat) + └── fig_kv_memory_wall.pdf # the explanatory figure for system_analysis.md ``` ## Candidate claims -> figures (status) | Claim | Figure | Status | |---|---|---| -| C1: 98% prefill share + 91% intra-session KV reuse | `figures/fig_c1a_io_cdf.pdf`, `figures/fig_c1b_reuse.pdf` | **needs trace on dash0** | +| C1a: agentic input distribution (p50=33.5k, p90=101k, p99=132k); I/O = 142x | `figures/fig_c1a_io_cdf.pdf` | **rendered** | +| C1b: 79% intra-session reuse + 0.8% cross-session | `figures/fig_c1b_reuse.pdf` | **rendered** | | C2: PD-sep vs Combined headline numbers | (not yet) | **needs re-run without --enforce-eager on `traces/w600_r0.0015_st30.jsonl`** | | C3: decode KV cache memory wall (time-series) | (not yet) | needs step-level vLLM telemetry during PD-sep run | | C4: TTFT stacked breakdown (prefill / KV pull / decode wait) | (not yet) | needs per-request breakdown.json from PD-sep run | | C5: cuda-graph ablation (eager vs cudagraph × Combined vs PD-sep) | (not yet) | needs the 2×2 matrix | | C6: prefill stays compute-bound at 95% reuse | `figures/fig_c6_roofline.pdf` | **rendered** | | C7: cache-aware routing is a larger lever than PD-sep | `figures/fig_c7_routing_lever.pdf` | **rendered** (legacy data, footer caveat) | +| KV-WALL: per-D-instance KV demand vs PD layout (system mechanism) | `figures/fig_kv_memory_wall.pdf` | **rendered** (analytical, audit constants in script) | + +## System-level argument (`system_analysis.md`) + +The doc answers: *if prefill stays compute-bound even at 95% reuse, why +does PD separation not help?* Six layers, each pointing to a figure in +this directory: + +1. compute-bound is a *kernel* property, not a system claim +2. absolute prefill work after cache hit is small (~hundreds of ms savings ceiling) +3. PD separation relocates compute; it doesn't accelerate it +4. PD separation's costs (KV transfer, decode-side concentration) scale with workload size +5. **decode-side KV memory wall** — quantified in `fig_kv_memory_wall.pdf` +6. the DistServe / Splitwise assumption that silently breaks: `concurrent × KV/req / (N_D × HBM)` is ≪ 1 for chatbot but ≥ 1 for agentic at p90+ context ## In-place edits made for this task @@ -58,7 +76,7 @@ old `--enforce-eager` setup. Update them once the new runs land. From repo root: ```bash -# C1 (needs sampled trace on dash0) +# C1 (needs traces/w600_r0.0015_st30.jsonl; ~1.2 MB, pull from dash0 if missing) .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_workload.py \ --trace traces/w600_r0.0015_st30.jsonl @@ -67,9 +85,12 @@ From repo root: # C7 (hardcoded REPORT.md §3.1 numbers; no inputs) .venv/bin/python analysis/pd_sep_paper_section/scripts/plot_routing_lever.py + +# KV mem-wall (analytical; audit constants at top of the script) +.venv/bin/python analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py ``` -All three default `--outdir` to `analysis/pd_sep_paper_section/figures`. +All four default `--outdir` to `analysis/pd_sep_paper_section/figures`. ## Caveats / open items diff --git a/analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf b/analysis/pd_sep_paper_section/figures/fig_c1a_io_cdf.pdf new file mode 100644 index 0000000000000000000000000000000000000000..d7bebf2b43e3a0930d16a3897540ba1d1c82c485 GIT binary patch literal 22776 zcmbun1zc54^8hT;U4k?hDe2pADd}#cyFpS)L{v%y2?1#dNkK_Lx;sS$LAs{{hgj@bt8CcL5>LuLJ_x_7>I-E_NW~&#&&TR@ye6AS0kz z899Iy8y`=QfTA-%g4EBa^v|av$mCz-K>lO^O*fi)cN>7ZALI*Y*?72mxmyAJpy>zN zSGBQru#j@~0a^qD|BzrfKO7+hfg(UqD3Tw6f(QvBfQAI50TzLJJV2-)gi5-&xB_nw zbfJHX1^xLSIjY#W*m>H6AV1rdb8rGU2MNeI0pgIcv2wMxK{Mdt>271;OyH9`kZM%I z*pVEvF

s%o>yT>V8ahw?!b%fTUvd(xngQ0~P|_)08HESRVXxN2l+3-PkSXk|YcI zp6)|0FR$Gj;_r@EZm6H^eGv)+l_iNPjG9G`UTD_qcI?k@8qn zLP`U5kmy=YryJEL@9Fe{q9vP`>T*WWPJ<5uT z#$Q}~s?lb6n}o}GcG>9tXr*w5R@iK!s&{F_+TmicUhwM?rssP_A9Sgi4?k*m7u`!* z*lrtIY)W)Ze0Yb^^Hr%|Z77|K^XrzPT#=dZ=X{c}Vs9RpDTfz)n;JEKJ+-BFSNO={ zAib(xGSbes4?5F20wQ&x`9e21j5@*zIXc=AJGns^yr)cJmDVTzj=lZm@?zh`r!#)u zfnm>idZ_ZZ$Ml{)yfZj9xjkm4uKpdj&u%8Qmrgm9pUQc?;zQWulXH*L9`6_%l{WSB zeYCxQ_h@Yg_pmYJ-uI`1!8LMsHV&SpYmX@p;WQ>=khhi_?&XN$RwJ?nt}Jt0r+|TtMz*-+qU*xNvGOZfH8oboMc7b2EW- z^T}upczE(IL?n*sAug@2jKbs|R$X(|m)RD+k}sefD0QW7t3X$f-yFt;1%-m*_iD@I zpJ7P*n(AAPm$I|CI;)LtV~~aqeKdM-vuO`Uj91<-Ou5(M3!$(6w4MUlb~*;VXH$Iw z;=@}~`I#q&i+n||$*0#6Twfn;M|sLzJe#6T)JhBGCw z$4@pFpR{*>rnL)qJH^-uSIv2D0n^KQ;9d~+cTj>=J{<=T z`}sX7VY&C7^wZ3f`-@4;@23wfPSC*Om%}Th0`Hse<3eLXw5xBp#a-OrKPMc{Jg~Ld zj4dwizw?#sZ9qQ#{-#9abH}K=6IamH!Y>(h)$MJ4rWP017>%VQ5*Bp>pgW~Mbyt znL?|v{DxUp#sSU<#Bn9T%A%#PQC*g0B*>P2sUTYtXzg>%qM+{pCyLC2S;8Fx2Zwi6f2)tTAV zYPk;XKEAyAnK-7vO=cq^X7a!+EZ4X}PJd^%;fYZT!Q$100GEsVy0y-N^{<_)W~geZ z3hj%Fgj8JXl^Qoqqf5FFnN(ORBHJdjMm<$in^KKacl5ePzbP@8206Zrd;URr1N%T! zez}5+d8Mkwm3`Kwj}UuL#FT|4&zVbR)o5sVM!@-bhu%*6pST}PLu@-9N?Zdu7)`y#Qds|cIyn^5I-FW6_ z^diBH-?zF8Zrc`XJfPX_C}@9KLC3Ro&CEogL;7llk6yPp9bM4F0vcm{F~y2-bCPrg zL4S|t3QK{Z^N+f5MZ(?$8?9nZ%rjZ(>j&`SJAe4bzFy!taCcKtfOK1zj?wZtI z4Mh1dGY|QY7Jee*E*X=2;1_ah6@kYeZO%})^{wzO*`y_FA~K2DP|r6Zj>;y~b`$UT zHYL%!avn;#@50b|4)F^mevDCU5`vPp0 z{mp&2h1sL)FXo0(4jFE4D3-`(KTABDR@(LGxTHIqX_UpX*J{}UZ*zvws93%)AHUL! zAJ`6_OS0j^{-W8Ks&_+=t;kO=e2SK{4y)iYVXqe129B%$oVC1nxr{ijYyGMamQ0YK zE&dIS@ms+td4~lM9)<aiD-c^yc7uN`dN-kKeKR^8?L883#V&C9q1EYt1R_J_jmXgV@+F+Ezj$;-K1OtYw^ERG|pH!T<1x5S=Yoo_;J7Q^b zftbXBF^#%iRm$nJ4mzKqQjz#p9Bc&5G^%>I5&ZW@$oP{FgNGC-JCD2Xp1ZH*%fuPO zE{AePtlnO|*_wp2BL}wBy_HrPho3hO>%UA~nxNr%4KFOT8ag*Jn_=TypPh_Pd`+n} z=9=4At`yA+40N5NgvI*jU=IRguz8EjBW^&mFIRUa2{{({I3H)KR&xog2EquC)?ICv z*{GOF3Y(iN^uXTlS?pzX$E?7oW$$hK7wqU21BP((bqE2&# zqe{>Ecv}NPdRVBBqrW-GuRff8BWcbOI_a3|7O3Xq5^uVRDpsWl4|3w&9(+&}(U6`W zpx#odtH`57#nDem(aG(w{%RDzT+lCev{N)~lv=h-$9~D{CIczgCALkcOg&t)x51iS zY)N|ZC7Tw_H0=79Xh_?C0$E_W;q>n>h=u6Ruyvm?ZT286dnT$GD1A3+lm zx{tKvSl+U3d(9?LJuh+p8DWC76{VtC<+-48tC#QV@Ks3=p>{N34B&0@TNz--pl-IR zsLUJQdRU~8L`|A>6r3sFN|IiYhy9UQV3MDs4fyMe1H4aIzx*t51Z`iy(E0K zXvZXK^EgczXQ&LjVXE|gjqL?&Bywxgatp(1Ky;)qY+2!ze&hvmm)25_A&Mh#m?SkR zw6rQ!r=-m|*t|^m3Vf@KCz33peipW6+%C^vr#VAzJO|$*jUfw=qeu-t{Em>N$4j0E|DyMt9P>V{|PO8E$#G^P7JscCpp}GpmA75jdC-Em> z@U(~9(FhiCL|M{ze|%M9ZRMbw*v*O3Xc3}f;AbW!B5vIrZgyz@BxWFTz1n(~P9~P0 zRmER((7tobuB8mu0Ng^;ZSMTW#nZV#MIXo0DQoQE${?7)AA9{;l0+JOE-Ntkbp!^D zDQ^;jJxjNFA=JNXH6@m4EqgX4o;SjD1~tpfH+zL{&TER*r8?3?XUadUl@_XrD{eoPnp zuXvIEV*+hV2nGvx#*gPrcgEhi_4_eI36eSY zp|hI8Yas=)=XWMR6+36F_$pl;{aZOSl=6Lp!W>Gs6sf43t@&{mq0LIw_!TThhr$hP->3yE z`&94q%OzVK&*!5TGETg~Iz$;2!LFCpsFyhs z!IDLw-ax0PqGae7#JQLf!V0$V*~m}kB~;caVdn|^T%SHngb!hGpaqX_9jC>g}G-_F{sAasW~ z)ne0grQft?)_oCw@7>rTZ@{KwN2Kf{K9Q4}S6b^CKLQ+12 zwPbGQlM-wwnY?^Rz%P^^KpA#CvYr*mVE z>#kMXWCwBSpL+qjX|MC{MRHVtT{*BCvtaN@x-707Qx+n@%5gV1?|XCjg%rH|JLOkn zVmf*4Dt(ULb!ZwnQC8MI)u-pqe7gh^tC$-e z5B=lt7NEz8^z8Mv*Rb#+)Y3^sdA^=7=pW(r$i1?HS-EW;5;y4i04{| zn?Y`-r~wQwD{AgV0ju`;lHN+qF`^;xEUV)PXYBUn^s7ZZ;ddyzF1g{fn2_9bVzqN2 zCk(*~_veobv*L(5-p_iQqSQ#KxAt@pyM#@k&BSK)1$X@H>+E0yQ;aT0vKhMc(8K%K zS5azQIU^UtjqkhL60;O|6Z5}VG_dVXkx=Xn4bGOgz(UMIbqVf=i{^1_ge!EHsJ!mT&Jfukc7u#(;f_V2+5huq%SY)rSSj%t9a}fKYrc z_rW_vXDB*T<2Gv-`8kY>6?VLW(u9{;4GZcTN7RZSa0M*oPoQzBt2_gWDa z{ct5M!p;xmR^A-H&7{ zAy!fXkB`h)m9&tm!tx6j_MDpzH^!Ei4;r^ZYOr|nQgJ|q@NsluIRh) zG0Sn!_}?IAxH7J&DLOAMyU=B@gx{q1G@ftER%1`v$aR*dZSrG=sbRk>?8tM_grl;- zga2bkLt>WC{K&yGk(t}O#IG6!jOkjkpMNegs1LY;DjY`)#IxHt2I2%V?mBTFU!V_J ztoN*&XdF8-@n-JYw;*wz5pQzhd2Bs@p>F;{st(I%g^$-dc@Dkz%5EG>Ljw+xy0=YJ zU2{_$dzGvaV+Ou8mDf*_LU*Lk$FFyOflhvXmiAb0U5&hrN#51vibrw&1}CO7y-!!( z0RNQ+CQgF}-cSd@CEuah&efrHiH?^ok@`huQuDQkg~Wbec0=!1)!z{d^e3h#tChnK z?c;rO%e-=%6Vty5PW$M=fvgqFEll?SSB`}?BU4VRx7}B2^xS*xXC6fKBAC)*WhPX; zvX3NoaT?QNNnH-ku{*PcB*C9KZz6+(qaLa&44B3Frq)b1S5a=a@G*nfQcSwb;|Fe? z=*AUp1Srfq>uo=|Lvv%MSx|d>j*q!oweXI&Tdi85h?uLi;?%0IpB6V`{%1GG#lUH` zZd|0Fe*|5{TV6-d!zfqz&g_`U-YO;j(%cVPI^eRoisy9=-?w&6s?IY#R;|JoaizL2 z*rx87jX!chj_)eL4o+&1DLh+cG+_izuZ7_=dl4%oz8)#l+lFUcO(535} zU`73Idb5~_i~Y@`yyk#}4L6fna)+A7n=?ziRi<>+u7kuV=4zzdljXWeMjw?xm#(8K ztcG2jmfN>n@O_U+T4J7TxW&vu@N3hKW^CRGnsNd~_#{irE(=lYOu16YsJ@erw3&$7 zaI2a%L0za~@a-bRNmy~*FWqpV!E@2+IGDm~nk{zBOSwnNZuf~L#jXvkHN;w|g$%_n1gs+xV>dIMF^=}J&@i!?KUVc~my(mW9i=*}O z2%#><#O~C&C$nqeDG~2{r^PbVV-c3|@VJCXl2AEXAO&smbnR5hoRH)csVbsPFWc@>x?-zJc7UfQ+QWrkUsG|^%1d;cHf3$SRxC>|zw*V=*>uo1=uIa{ zikEwERf01G=Q9?acl2~Z%%+Zr`gfl>5SU83gK#_Pf}Bq(3kCmcnjjRm;S z^@^7)?|XSaSxAjz$sblIb!-`bQU6YrJgm08Xvms8Cjzzc9xj?8O}}fQqRWIM}Sf0}0rzBxZ4 z$Vfcc&vo>^>HUV=>&^F-3tXZ;CfKD>cG`3_9K|^0wz`#!>T@U>#o)as&($ae#FP(* z$FAZwb%s@a2;ZfVO(#3@DdD%$c`V8G5n~jh#V^`7&0kA$Z}suhg3B7C`MK8?i>o1Q zO!Ub-x9q=Wwkud=Z7QX)j$M z{Ma0;YFtPv?kG5$yYJe*SRy=X29c41XwMGe`&ve7^;BG8-sJjBHx<+FA<2t5)Uq|K zqGv4W?fcvoL&yGSAV;g%j`3=iojIjg4u-lVGvcmhd3rXo>7damp1IVj@i>%2$Q{xL z-|{EC-QJpnm=;QW9F3l&fn7A#avkZZ^9*|w(uh#f{yH_A6SkheD$piP!g3h4Hy!xS z9$uhRtLmMbZ{dHv&wYrUn&E}25`p}i1_z@DboRQu7S(TgV48|;clmk_2unN+`Bq<~ z*z(qUg=xk?9;fdpKj_`W)RCnhu*Mr*lc;cvs4ITK9S6NN;-`w_a2Rr?{nE3#&^VU* zc&f#In{Fa{Q`|2kXQJ{NuMZ1IYC8`-eDlj&OLoc1zRKCkxXJskJRebr)9$jr;}im%7J z$%`7C=1>dJmv*kTK!5qZRutA>%{XD+-c`9^z)`33%K*FH#e~yYy4@uBKBaq2| zVCeIq-wsMp5a4j+e;kq^BI<~n!FV#a@~X-Ah5YxzvMzi*b`IscI4QOo_QlEEI_*Ad z#RBzEw%g`MAvc!8AVpu2ae!9zWj7Mq}O+;Dj#FUYO=-Lmr3l4w_7yx6DM+I zusnm8e;;xEJj;Q?5Pw0VrlQL^2wvSX)v-iwDd$G=vyYiaCoe-Z1u!w+zh~QMkz*5Y zkN$jF&#(6*yl+#);6ult(%grJP=XxpRssjyL3dpkQ96C{`9#>3fvtElwFtF_YWsEX z5AYz#PzPiAzE$v$=n8&~IL&pdrv}BA3ZJI58X8BpGq`)@N2YkI)W(LDrn2nT6pg)2 z)tw#8XuK{6V#km$i`3&DTw3Dg7mjiH`k3o}d=85cpFeY0=G8%x5^|0^wroDV>1rF< z@cOTkL%R*PB;{Q=thuW+sd9 zW@6DTc=6iyM%HVFUR1uDN zC9FGms^=dyx3{GD5D{WLwTQ9epV3Lsc^Y4w5}7{QU?us^ArV3BJRY!J^zce6Vmf=1 zZjDI_x9qZ~i;W>+vc9Nesz`r!|MDHRi5p@xMR~)8g6?CZV#0mH8m5Ma!LRHqjyOjJ z>L8}f_b#CD{$v)4KK%PlhXs-Un^zb@PzS70T5*mdFJy^eNiKnaE5ukrJT$%!o3an1 zykDi?pI2JhIw7o#CECW)BBULPEc5TZq>m#(=cG!~zr#>WR180dxJZkA22KAy2mMEB z;e!90C{;jd+rdP2hg9F2&m|Og+*E%cNxFVg!p?YMy@iZb*?9hQy9~8yrN*e#{e?C; zQ~bvP8$lzf`*daVSH|UX;;&pa0#QOV**rA`>Gl)^>j~c6$qCeZwx|Dv{07f}k+64J z(_Tx(QVjQVDbWw_8O9V0M`hk@iB(K{NPG5z0>lMbaAh2#Oz-71eQon{hS$ABmUBJ} zlFtj?RxGDW8RD|-cFON^=GKK02tJE=aClYDkaQdQ3hMu8AsA|??p?@{^?jH#G5R9q z^;Uin5o$|HLi+m^%<_2uW%8kA*c?sT;J7ZVhwd`|pE?(W3;|%$s4{Lwx zQ^a4)ORyL4pM4sVa7LdJ`NQ=cNK|A=^lz>gx`UL&ArjdWsGy#+q0)jkr+xnLst#$-@o0#jZwskWcQrmW5p(wVy?9L5z%ouRa}!fcDk*AO>kktm z7sHioshd(b3Mk*QU7wc86j4=5Bdr*kn7y}*xx31^b_Rv$n*+a5DEJo&j)-hQQ;2y2 zP$+ay<7>e)@wZ1!d@_D(FuXP27j8K!lwZmH0*+I6-U?|{bgR(XkC!*TXR8THWR2rY zdHdXQcUhKnin%A9hUg5U(DxU9BMJ)n8(X2FT#O09n^?ZkD3L2cfw!Fe%=hv1-9vDG zIQhsd^VX*%p^!x0eTseEdYi)cU~RuHExsIY^G_09bH*L84+K%Bp{i>Yv&NM*iaEs~ z=}Sti6YkQ98Wmi^>~~#LjB2^6I8{?zhI*Ta`=ZNPLur(7lr@H4+P7w{7wo5PM>GkC zU%b;r!s9tPH>6FzE7oQw9;)2X`89tmS*BK?J9;QZ8DAAX_GY8|!Yfh1oak2|NiE|* zN1N!Q)x>N}3NWOKSHfE_E0x`$>unRv=j}4Hh4IzMGvsrIxrc%OqSwS?m0d6)MACcW z_xQ=0zm27T$+=vcG<|o1lYR98!x**fwS-y~18t^g_@m&vtDA0RY1<-BgSNJ_hFOVW zi*d`ZsVqNd`Hl6)qtLo&*dZuFJ>O8x66Ov4C% zd_hmWY#T1P)w9F9Z*P5%fX-f1F>%s*ev4VuR7PiI1y?@ol2zixSYmy7pVn7X&~Hy5+tJ4y}im-4XhD7+7W zxpRKD?_3)0o{v7Bj>J-aSZvSoWwyj+gx|bgTUgf8TFt%CAzVjoPvgQ+Op44~-2Zwqs?~b|PlCO&n%&Zh^}GON9~C;eAB)K5xMSz8_nlY$o_uT3tm1`$ z#5q&%z&`p;n=2$R=K4h4S2>-Jh`MEwVxr{G_v~cC;5rEF^S39H&(CzV%UYkur2Sp|IIzf$C9*K8~1XjWXnR^hc*m z&dmDD89`3s*Ft3yFWb$AwX|G*O|n+AZa1&PVsA3_)Y&RuPN{3CiC9%N7Vkg?->wq= znfmzdAmA_^lB%&>B|k%8XE-r}u>UPD7!nCg#9%m9jGP<}L7s%}1D1;l&9A#c!_hzr{S84C z8UR6pdan_1z&t^r0C7-F1;_^|9F7a~s4)*jnJ0u{L|fpowa~d&n;UvG3xi&06rqf1 zjUCQi&!Vt9gO;-#4A@_oDTF#I7K??b@8&&munx_HMAOr!mB{x8cWYyl)uK09dWI+& zlYNp|SxarZCJe1XwzJwQ9Zg>=NGYG~%`#nz3n#P)iE$e#TyRN`x_IIpKd6tpf!u#! zU`qDBRb@nv+Kzt$+sYxe9o6F!VM19J=~eVpTVEcWNfhi^*%k-#xSF%m69rvb5sKkV z58wDY!Rw8>L-L6b8OcJmywB|$hz!bFySZ_m&Rz1vWNM#Dwr+(TM2}BBqG|a#burki z!P}GR#Rv3N!95ni=z*A+IwW3Md@)zdKJ% zLnKOuh3FINp1>yCHme(x+;cwLjhSN16fO;wJC!iD*W>~6M3A_W1@@vR#Q{y>Qd=(b zpMqLJ&!x3MuY5^EJ=gM(F_glLsjw>+f>u$@vJjlqRnI&zTn^M>D?zXaz0I1g0%zE> zOHm(TQC#WVw=#UFdDpvxT2rbD_0=hA zMbO4UKQiTcU%Fi=+MO-DVy*hBkTrR1NPQxCpTVulY}{HczRQ!Te?PV2RS=4*c5fjJ ziK`kpzOuGQ>A)?ZRmW(UcE^h%eu`$9h9c4KF6|7kOvXLFZ+Php37+8>!BBr;Tg9}L zT{JMEVrL_vU>%$2|tk z)0%ycix~Fv#O5x5Z^v!F)+v8Ys2_Hd;w$RA!v6Az6HAD&%gA=vA~UQ5O7Y5E8D*ZC zxFA0}Vc$i1En{?zeOh5n_-FD+#vA2@l4HDHaXY;>ST+9+F{ zHqu5iU%+78EP>;-A=f@hc$YVRajVR@0$a&)KBG_1a5Dw}VwqSDu0+phaIiKQE~o_- zL~4N{2=t709zP(d#u(hf0Pds7=f&j>HBNbd@_X+gLZ%*nCrz9H_WM5f z>F3@|kj}^=sq7X~&9t~IS(4P#)4&HSlw2H z*5oh6vrwuLc`=X}=CoEjTDHv_z7dU_bUBo>wc!XRerPQ^0ng72B7+y1P7wtQ*CTTq&Bdp7^});D|IMDu%Zly~)g%MxX3 zHnN4h8H9&*a#a$afM8sm;}PICMUp}N%$G?%2V)af-jrj;j^{9?(#KV@n$&nG(J<%E z*E5Mt>c)9@{Poq;grOS_F^W1|lQ|SOPe|-IqVsRQu}a$`v^r!JJ%h?K+&dWjFATCQ zFg?luOOGaPw*ZB?#ON;-&(NN=(PNc&1GD(%PCk9wdDgw9HSDZxqjBoDFG&~XeTTcV zUCP^6#K-oIf7s=yTo}0)lYb#&d~2a>M=o!S%5eAdo7{mc&4J!-R5w{>I#un9Tf-z{ zo4&>hwHt8*EDU`o8UegEQ|<)pl18s@tdY-lD z#qUpDxZK!6axVWVWd(LO?f7RTm;E`N+Hl3z>}_8c?<}Wmoz(+|b1)LHdvhqe5o6<< z>2nFUO;%(h9~|*WXc|a-?|$dqG7afbaHC$riE3~Y?MQZ07V11N@t|Rz#&?tb{gQxQ zacIgWzE`caKz(ca_|me}BfJ|$=SWPg+lroN=O?VMiUxkQlXaJi7!DT_W{EX?iwc~7 z`-I7G!C0_t-kG(mMsMab^IWQc@z+lmmDhsX&*+^qd`ZM#=nxDEy9!KnWmmyaq$U^% zKW|BS?}9Y{6TE+Ri6ox6S3l|XFb5v!)jT#03G~m_;b%CGf`8%p!=P2bG+PSCpa`Mn zAXu`+Io+XZEC{QS;=ias)ZU9xPUK&~o*Ecx=br>}ChGuE%akRD(aJEUV#M^#Dq)^u zpP=qJgPt>-N1?y)ti}3K6#ig5?APhq-&upZFYR4t2=_VB1;4>zdhYv#MJTAYrD)E) zx7L$#Lwn&}XDS7LzK@b>LM@^1a7E*Aqy22%rzPcgVOItQFQ$>!G15+*LE0I9+TZM( zohhmWGF|?QD!jP;=iel_5nM`diN`rtV@J$+Mzd!)H&Dc1`1Z0f%I;VoJej@pKvw_n zA4DH1*+6Peki@U7k=}{59-}Uy!|=)T6UfcK*4D zc6+j8*tJhqHXo{oL~uIjUzwl^tgmNBcwP14c2gOQ-_@Kt@HU9MEl*-Y@V?N4!D+VF zxs8TX?s3s)lD#mHI;XaL$2w?i6PGjccKmZ5w=}1(ZO zk}H4Nk>7BXR(Sdmtm{MGVaTf*^TfI!m5qZdBZJ?LF`2k*znvkXOx#w#2-bt%@NFNkbM;SOg9vHR~c)yZ;3VWjn%g*ZiD$)f( z@d3-rUpHsLK2Ga)O@~9g=axOvgNLw)lF^pLH!hp@37Ig}^m^6@_q@-q4y4_7&32nC zr%;c>0(yEG@diF>PaYsevTAF4j&r&u3rFwptm#-d7%2DQSEbe2O5jgwqsE_$U125= z{(1(lXLu_x=wFO%gl3&{2bgFLfqfASy2C!O08)sly8BpUyg*^7o>|8fnrqDO)x&LI zv7IaJ^&rnL-8Sz#$!_#Q64Mw_xS z%`#71w)RMC4Jah&E_}bRlJY!3-|{?h_`RYrs(IZhF8%7~``&CdxcKf6gB5KO*#}%z z&3?4JV=;!QpWF=yPF!xEr?a>xv{oK2GkdccQ-$r=+TUqV+-U9NM_HtIh|ub5|8W1u z18f+Q+CtC8t1S+%)h#V2U*;4-jg(|T8j01Gl(nYyYPtrWNKWs=MD1=wX^^C7;O$JE z2i>1)mJ1etIoQmPS*cie!%FEO)tajj3rX-bUz@Ra({im}*Y*APu9aR}O{eJi-8ZWw zy^~KUEty=cGmTzU9tx6{-#)=GqrGK)MpvBWXF>jA1dD0ZL}EdR`b_T8={NhX7uZpU zE9af;C>(sf^oCkiiyM&~%O&6y$8MlKE$o-jPI5D3(Zw3c&MhnOsFZ#0HNn>Hu%!!o zQ|}VWrQwP8s5^u(?q~FjL(0e$H*U(>CzEZkZ^X`?n7a3w%8Wp-bIeC8LGE3oG(;`Q zl;M|%lvHs}irpV(n1<7fw{1yZh>0axLY(LyZF>-xXg}nPadB;&H%%=d!6avhO{r%! z(J$ktAa_7Wstz_A^7+?QnI6+uu+|bud<|>Ao}v|%+2rV!J1DTosr~*85uM>k{mry5 z`=7%;bF4cYU&S@!TBn~UW(544ARcW3W961#)H$(^>Bz}9U40iL6D-X`eEz+tdwZT|9Z!#{t09Oi( zI@daLIb!!4&a>~Pf@jF>48I0~_?v?QbreY(m}ugN;10;&OTTjZ(WRB>QXMRaR8%CI(!F>+q#o+67yM#mZ;sD1XAh zrrdvfaQCyzhWBj@^A7Ooq4clk*Z?Pg?B_%N<{Sv|u#XsEPkJLIkPz$j;0sap2T$&M zo;b2U#U@Gu_i&C;2gO84tC+7PSvZEXZHcD{K4Pfum?k)?kK{+xUq5}5(zP3Qs_O- zbM5_$f%r@R-ro%3AwD-(L7a3((`PtZQ21Y%X;sa-BH$?Clh{3KA>bsS@g~2IPlwG3 zGA9+QKt}q4Vo#fPt#z-(Z6!H{D1K6BI%TI?Q9&u~`ws#O9K$}wJx|W-*XC7y{bkyq zc!_5AmX6@?%ucB3`54VW9g6^xXpw63{V_)lCr!djkPr=+ze6j5M>Pcmi#Fn=9ZeiJ zk=naKZoNZ+r#A1oOMSp;m#(}>v43h-%i!~g3s>s>WPF`3iz?pdNF1wvUgH}DFVfC2 zsCXuaMd;TPcMY)+5u2vq6uON>CV|T6S#wEEacPH2Z^$xvRncrU8g<6iy&Jr(d3(uZ zof)r%o^$^eMlZdI=^3&+!&`(P|6lEDft+7}&O9m{v` zmJII)NV{lz;KuVUCZ>bJ{+@E&PJyu-qTgKbzAI%vl&sJbkuKIoTvu{^uwqn*=M z!1jJ2{6xINgAa?(Pwut9*@ujhsM&*9YKOff=b>WXQWcSBIi(Cv>K>*XW`=e(!c2p> zJztC8lWWzTzi_Vx_R3@K@{q1v1+_O);hM3NYt)gx((8LJ#&Qo>qcaLFi9X5c>E|Gr zf0)YT&W&mDF{RkkonGhq)AWAC*zC3#t z-Uw_AQ#MekfXx@U1PtCPHx9E;eFX9i*hn|#0--AW*#Z$zGhTaTfyk)?NEGTqImEmIY~(-Hu2N#GO3-A=2%g2 zc}S1{8Zyi|3!7t^DVM#!{ep05*ZF$ely^ES=B-Tkxgzuk%k!C>wv8y`a~O*jXve_Q zePC_N%(+tR;U!C(z|V^dh?3UOb}Tli}_c(yOa}_s{5vGyJ>1+vETa z%+5)fD6?W)fhH!qwOBf$`?`m*`cKV?n*mLb&Y~NwghNu+k1H>S!6av#Zvw03KWy3G z_}{0t%)jk|G;o=QaS2oF48G2Ca$tX9s$?H3lVAb2nf6k%qMEyym#;%t?|Gc8`=_~3 zT8Z7e8c-9MDaP-JnMTFTLR-02Rge%RYao8rS4)?)XZC(!XCKSt6W+~0MeYz5jqoJ8 z7(N@9D*p*2@rxJn+PP7s;o0DaUn`ubn9Q<_E_uaX32NsSm|Uo3?9Lsjo6ZVvyqkjo z!T*yRz)t)jCm66p^cTYgYhET4I z)5}gu2C&B_p*m_kcV_lqmpwUtA0gNz%)M%4XkhhLoPIs7lNY+`tH0y`1>7bwwiCB#92Jt4tf}i$CH;ta0Mf`?eVvBya}% z{vDoCXZ;TnS3!za8i-8qE#Doc_t%r~`yY*z64@4#evjaqTudL`7d;XXE#@UWtg}dCaiT zTn@6De?OkJu=mrmsk`gkID<+l-x}=1&ys@=*rG(D)yhKHWbKe59X~uSHTYhhM(=p+Y@rmuB#h(ONJ(K`n#47uWFv zM&Mke0Ly-s0B!I~LQ?KS1<&^~#CB9{lx-!vo6l0IoIThn#|%`m%O(4Eiq>6&fEW=| zQ49oU5O{`L^LMwmG*KkL&f(e~aMPD9Lcm8o>+-;H(?G-=g5Wk6yj%W@dV@-i*2tg?yO!;gkMrn((K5fQxMZwH!a91D%Ha5s|-0{{M#Ga(Q_6Q==090KC_pj}BI9FVLF5S9lWkLbT~kp2M?2*^0t+S&logwWxG zj6g78x7EYV!U~9N1n}pE4m0FrW9#`d>+WD@?+JnfD;6tPXJ-qLfc;H3dm9(vtBt#Z zt2I#6$-={)K)~0=-4!I@>H_fTc>~CK+Pm8Td0QY#5%9l*H<0%LSP<~A@kW=nIjvT} z#Q|Nw%GJr$1%!kU2-sZrvTy<+fjbeBAS7_B4v3K?13GogK?3q1^xY*z5c+1a3P?Z| z5GzPP9fShVEI=q=p0EN50Q8_61qs*@K)^sP4nSZd;JYJ8zzHPa3=#lZ2555w30wyW z0CWMJ3-IX$fS95cpqpyrhQ}I^Yuu3ebjb0sX=UMW5tLTevCw5FnU9K>wdCAR8dim4lU} zi=C4VK)8Umr;W2NI-_dgbIK9G5FGmNSX2K@xJE#{D0IxI|KEw=|7#sEz|n;RstpI6 zJ0X}52njnxInw7d0^83DkyZC5Q3=FCkpThkN_MBfdUNvXcrE&ivT7P7z$ku4nhD=AwXE@ z=V<~Wf}w?h76ST-ZvV8?&;kNz00c^aZW95NITgx3#G#+3`h#u>j(!27MT90B%^La% zq|^RDH}@kEAb$4Qf2#VC0SNq21r*>ID26~&1yGJIj#h(zy6c}OKrOI6{XG66NA-~Uq!nkX=s;Pl(i3Q$NC=%0jEgdaM9)`MTusjLCL zKg3Nsw+Qr0<91KPH4JL-+?;Ny75m!bB+WffIb9hMeD#B2_*#d30f}z z8}{$?k52saL<ITaAl$p8$9Q3rzlG>~c_puaaoEu+!Rp%!+{F{PA{bO#ntu z-)w=ggw7HCFh_vC{{W*c6fhW0U+h3Y7oFyQnmOPE<@D77P^cd{M-Xr)1%MF%CIy&H zPro{YfPLT7oD14i{>Y(C8~i7iuE50h1B|wiz;XO3iW>+}6d*_NFRKe&Z2IvHZHIpt z-_t#OfZ?AO_jJn}c)bpcnjg6CAjD4zdw_uLiqrBx>@4DE`)Jb(OjmESU|w}Ki~LW z-R-yt1OPYC+RF+E&-m|RZq~LSOA9MUpqYQF08mdGaBy{zK}WT`EF%mBuIPY)nHU0r zLEs1;un;F0%*pjH$@sY2*b)F!kRSnIEr0$2#t;FFD_hWiU}zT<{ReXS34;O^0;2jA z21bDaN&OB(kJVpcfR#A|228lW<3avt7r;O(;xF}q!7%gy{S}53L~HM_FfbGhjLlzR zNI~>4{{;p?0W$jo21bv^U-5+Cz;Angg+akEw1@c%9vCSI7@pr@DD;f+E1n>*{`dn1 z2PW{}@sQ|=>$kF~KWGLEfq{eU-|&Q>f67z{1~?(V;~~&9_pfaT384r1uP_wkkA4FK zPTn8%p^)f#^S82sfS2(r46yltWA!@>?Ew7_69OEj-(bLM>yP??CD-q=hd>bMc@zEj zAAN>E1ktwbR~Qt1$M$y^6g^Y^h6e}#iHAVXt-s+R0nhh07!3R;Z@|x!f0sQ#3ED9I zS|1FBwu-;P;9#NO`w{|2`(?l5A<-86x3Yjj{{ce*9^r3z2q@rC{|-Z-J%?ZG1ALD^ z@Q`5O$05JrA>n_@3W)$5t>5vGe~dQ>Qs@sI2f7mVN4tVZv_JDJeS$*3dh|D#5E$*J z{E7$o+Q0J#%zwZl^*?x??iLPCHtwepinSelZB7@uTCT2WQ*dfR6;@?6T@|1pg0bUisVr literal 0 HcmV?d00001 diff --git a/analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf b/analysis/pd_sep_paper_section/figures/fig_c1b_reuse.pdf new file mode 100644 index 0000000000000000000000000000000000000000..4f33de17b548e01edfcebeb17b079bc7aa7d5c1a GIT binary patch literal 19643 zcmd742{=_<^go_i*OW2Ja3#rj=X+gK=EzKB&U6iBc1cp1C7EX_vxH=-ghVBoHIbx= zqKN8M8kFBY7w=o#@3)@+|M@-7?|Qbg&pv1Gz4lt`tiAR=pLN8IHMM0?a#)yn)f0GS z6AT4M!hM_$z?78Wh`on>NpOUQBiYf@#|@4!c61{Jz|r6aQ@Dx>jO6VC-6;P{1#KU1 zG8{v>0byz6bb#bch7*=gwSvjJ&B%^q5*)i+VC+aHll;BmIOr6HFmrcw@$hzo<5#Zw z`#77C$Z#9bt)@00g%nJNBXqq03F<3Hjg_M=-0l~0Q0oT((G5}WPXg2}laDYZ1^5K| zJA?5+^n?BlNG=|Z>OR4sMeW07)1G=_kKqcC_mEDnj12i+hvzyLuj0dV3nnQGqN zKHv-mRr!@E(B)Mw>XW?P$nJ2|O3&IJo?zT?gtjMOh9=3`$AtuuA3*jeIeNi@^GYoY ze6|ucUHfNWZ)q4A+3|v4La+7&<M%xwa~MAV*?D>Z_^Zrx|HK7uomSle=m!d0Mj# z_DI4{ey;Dm-;V2>_gmDzym8NR4`qnj8j`6$I2vOm8=TxN z5Mm?YQ!X>CT_Ca7#V6^M?4t>x=yLZyHfMMFZij13VplchLQlnqA1u9aNVdt)L{qm% zU#Ko2x#*MjfbX}I)C1eB3a@g}8_PT5(iuXdWc8nRDM%+8ZV{qq?4iGM&t$fp<;)M- zSE*$AsgxIP`)d4i>Mor<@`6FCs50`UntP-|_^_m)%;s+Tcj#8hJ)svzgcjk>akoSE zJlW*zl_awJy1ZjcR`;Dj$0t=`_0k7!Uwg8RG>{PF9vbUUcW$D8=LhaQRG9(JH2j3T zlWZc1OGP|)iA~|S^A9Y(cd$&mxb&FLh`4Wmb=JI*`Ga8rvuGYa4XGGyGM;bRlE8(z z=-y1*UtKQ!SWix|UUHJ&ugZW zR>poha8>s8_WG8`b1EW(Z;$Ddtw!55c0^D06wSt;esQ09Nn8PIG%h z@rzgCWt_7%0ZCl9%(@LPN=jaxEtuvpXp)M1FLX`YLdQ*&edba}Mb&3xPYb3qv%7!n z>2v&1gXsS5mtGar;{A**C*fo3R{7(kJ@}(pJBQ}&PM!&ou~TfAGiA8`@DO5@_po4G zoop-5s~nZnBINk~vpqqbKbh#P7fm6Xx$>Z|SWk*ofGh=(h+MU(6l(&N*}(1oErZPH z8jFI#pnw_qgE^5k)${K_GLL@eu`R&7_Q{_NRIr;LJ8l!#qD8wAzHuZ_?{Z03R#W$_u2O#D-Sq@Cm!vnkZ8?3ZS*?Yp5;UQi2mLi zvvAD@{$Me?8Aj=9r=#2PQW-Cq?u=;O9Hn(Pb_(wviZkAz+{>ecx3#e)NU@o!I^P_$ z!fsfw)~UNp)-x#*I`7*2Im6=Kyyc5oZpknDnOk=_mi8uZ`S@7;FnK1vSt3MH+VIT| zhCEnhz<_?6AvX~LvsO>>o|KSJ@P#By?lclA2RDzYxT$U!}( zN^&gCrS;Gcx%<)_!EDUa)exUA`9K}Z+V@Z7 z;;meCK(i;0*r{>7~{QL*=c{<5%^9u}` zwp|;o;iU#9#h&VQp2s7V#`ni6#29yO5ek+0+!Enolijq{v3gkcLGgh(nUY4X9d(k1 zjJQzK6v70%MHCl;hjS`cH)(Ebi4T3L^}~A-FSXeRX$0QI!N|H=O~(R|X&ytF&k7rP z?z+)`TNK(QMzotc^F6G%EtFvri*7WERzTl=@$P7QyEfXEI8@y#vo~u-}kpad#Cqdbc?LV9Q${EwvvO)Z}J-*twqE!{gNqTKj857p7}5*Ek!l2R_EeG-7Z3yHhWZS0w7 zFU@OHxwkS1xvkeP!tjoV&)!0tihDm!$BrEPNq0N_qUL(tLqi_N8he04|5?14%SdDJ zhegt+YQN@##qedqBx7vhG@CG4EIYF4Tm?3GoOmhNm@8P#T8-T>?Zq}irSD%|tQEZ{ z{DmpP)ML#+-xwy&7+2V9ZA#CoDkm~}7`S_A>U&R}3YT$q7AtrwD5t@#hxBrCFLXHI z!EEZB=%7j_`IgdbIQ@c~jS4xCW3`4AXy69^(KIR>>v=PvSRYUF2#Wr^tujzqaV7ZL z(&gZyAPjI~)q|YIvAeEr?d08G*8p>h&c}5+=Er)y{2F_N!11Woaik{5QrtH9U4+fc zOL_K5M?RTx+d<#EZr-^E-j!R_j2$p(|)~iv=B)|=l-agR*~?Pa%7(| z{Xy9Fk-o&D7%|4KHqm|GlDbMlQg~m#!$iL681>8Ee68Y?i?u=8bJ&|MzJA6KlidQZ zar+|wNqVj>m&Vub>AmU6tyAawv+_Kau6%R5HoSnSD=?6t!W+mLU&|YyJAdX)a!&#s zidAE-TAfYC;m5lWSI&vLAN%*EpAlr-ft=tjeWaG=d6z?4>zar9btRb{9`Zjfp_9I4 z+wQ4M`CjD4FYz_#-ld(7bl$ZL3?5FO@Q#goaw;3Ah0Q&=IpD34q^`Jp$;UgshKrK0 z%1z~=lkaot6VZu51*2UBvOC>5-n!+xwJ;4msCmq`r;0Bt_f=X;9m|eMTA#YN5jUrl zjRXBWIz|{X<-;2oZJ!Dm<0{(g6b=;Dk2$u%&jxV&Ha0%(*>|IgcS~FN9*M}*f?LtI z4|N}D`AIW4ZjM{eBowp+S!L4lnuoHGL8v05-y8H;o&GE0sKRO#fcq^(vN{7+24b*bdf_Mp{Eo4{hNc9 zEr3E}u%ZGh70UVwhS2f`3lC_$_1`*gwFrjL^l)`0fz=YUJhFjfz^vg*0&69(dcxu< zc@G~KI948x2qF3Vz!5&)UpA}m+bF~P1CKl%-2&hk^fD{{S8l-(Y5|mS5Ya?90_p}Dlq?zphNa=?t3&c| zb0nl~3=2pWi~Mx~y86i3!0|}1ko^Cj zVEwZ~_JkE&`6oV<<_ULKOjqfrtT#0Q4CT zl#NKlf%^z>c^s&V0fa$Ep!x)0xA5|Sc;GWbZK6;}49K9s1<(N+2ZKffQ4Jj-DsZ54 zDB(fF1fcjhBDfzM@dUsY%yJn4G={+fdk+#G)CEVtA}r_uT)@i{C>5YeSWp=UD8quv zlxtwNC4!FeSU@$%;{aup#y}&`5fHE}H;9FR2nuUKr+7ddg)OVpK}U)TLF2=MnkW>W zK&b-8w5mQZibMcMq3V=`AyUTnZ&S+|h(M?bBoR;s4IiQkP`RoJkOoi$2Q(f!LhM>4 zWVr=AG$cHQe~>2p+dgy`#8m7`0_QLS)Cx!tanN<>8g8WlARMUn>OV*uD2f0n!0M`g8B1I7(WmublzyaYv zz>sGAl3=Tvv3guqh-Jlq^1mjCxYg^B0<0!TIe-GepyPkbImisJ8a|4lM6K+TLBgm$(z zBs1NTemk>#wDITj6)N|BT$j}hoxw2Agw*+#=yQMO3=RL8H-AswxZ#ezfcxY6)<@f~ zMrMhnOXS|Gbow-@#W5z*b(WuX{p1S_@qe6HaVCj>;m9igLewVOTi^GZi@f>yYzOCC zCK1LFeh0^T-1GL4wyb-P3D-)r(jGHgT+$jdKHJPwYH7E-RR4xij^c4svfw*k_R(iU zujCEJr}i%l{-EL5Bh0d%Mqsqo(TMq*#guZYo;T!0EvQ7xZEE{Aa`vB+olV(KkB>@- zzd0#5!l!j0vq|5|OgI^v7Ipm1d*9kKV9`3@>iXQeC@bz&`s5(5(`-@bNUye?l|;we zC|89mQzpdCOn8He{7RY9F}Yy&;T>mfK6CDB87vTffRo!uAnz+CNn)KVz8}ALWbqXG z`2l@9Pt(dHB1-m}=2KHlyW_SwXKkJwSKIhz&kc9$6B&oJ=+tIEHgq4^I6M0Ez@0j# z^V$81lCCw=WK%is&9S1wcRrTCe^vbGp!&d<8fo!)okuYke~DT5+pizod71q4X#%}o zQnkD2KhH0E56L;Sm?>yExfuFadBmF=E*SINzk2f;>!wehS{jl8DG%T7zuC`nFw>JS zc+WQm;o`9!dh8Y>qveKk7nDW@7euUw$s$yEPi3)z#{R`BTPwxDpFxy4YT;}V;(f)$ zKIMTDQ!D-1>uh~3+oi3eU(?;E<=*|COJgQ3gIfe0ikL!&UO_9qR~X6hHEQ&>y-B)> zakP1^DAl8AuW{(I>8>+{6?0pNPvuN;WRgpMNMSBZZ}N$1PAjr>R-o$r^jXsvTRd$d zcs}i^k~mXT)6@L#w*nduj25Y0Yk%Z|CX>YbEZ(X4fd{;@*cXyu94H zfcH!KXY-fnL(i6^*^ds>svO^-_tTD%3iGHfjWB=lbm2^Ny>B5|o4@lewrv=iZIS3G zi2b}YY}(DktSWxFQQXN{j240VLOk*0ODe8-$1ykmeTycx5F{^ds?23dm;0z(=l+<61>u)Qg$+X=mOa=9<4JMc-|9 z`PrKYF0aaO0pF`wyuLr?HViuQR`(V6iOI~WHqtn6T8Aa0vdL9F)Y0U(o`n*HEj#lg zJyi~z(9GKD_B^h=edi$i%++^pFU>{W?Z(b|IhSki>AZiP&A=d)`K#tex69aBzMsbj zuC)$-=P(rA%SnY{tIiwb7Oz+%^xtf=;f#o>^rEaihoe=H=KQ0o*Uz21jDPg?cvEVQ zVe)&?uKV0VIl(z%Vl}Rvqt-5P*XL&X9oIiMaBx>FJQv=U9?#+!lj1v6HST>jar078 z#(*W$9RABmEBj54Zs-r~<8zZ6WpMV$yG88sK4r&Un-F;6oSEfrWFc!5`SQ(lc);G0 z`L0Ct>nZsZiL>!@pGReah{xEcS?~#>ypvy~LL%^yMKg!zXt(&QE!mBI5!Py+5{GZy z$Twu-G@JJ-YX7w$vhbyyTmkC2YI5A7eQ%`7Hsu;u)fX3-n)seqIX6|6fBe;cTUjdn zf>t%F^VllCu)w$e)3&6tv0|d8DC;ya8u5PnM=@Vv?MhkKxo6dMxst~E$1Y>G4|0a@ zW<{l692dV(P#u0fUj2jj%jw7)@JbC+cz+1T3Gz%CK80IBC=avCk>H%zriEh6dqXZ$ zX4*mgeghU2u$44pp(7Dju`Tf}CQ1DXCK6l|Zknd^MH9aMej(#g#7PReXmC~JJg((+<{wlGYk^4rYIbQ^Quk3&9 zVwllM7ViC$*U%qHX1NLYOuSS)Vx;$dxh1KgrLP<#t@)nak2w zrk}Ca+o*8x*Tv@w2Qk2n`!fek^}IV&nMWtV#_qlMrw0z3nvb=7_c^FxdS0p}4s{!O zv^)9gD?K|dzv!+j_SH5@WubC5);WhyG{+~1t6YAPH7k1Ajx1LzFSPq33;K+dW_e%5 z-NG;GnbAzihVKhmZX6~497lw;57)3YzEDQ1)ogz)seaqrL?D+-jni>5g}1!KIN$2V zGn3ir1vJ9}pHrnQ&VutB8yM8RbZ*hURo$RDlfL&ta-!RI?(4i`Zcc&@lXvTAss;9H zW*(X8JaN9i{8WqPEfph!Z#y&fWi4)dn+fb+Oh5G?l*}M?9{Vx<<6eniI?!DG4thj~!mRP#&qu8SIje=PHujUlMKXqb^&6&--T7qn zlRRqO7ue#@T%`KSK<#}qd$o4$ty^f7GYOI*MKU#8a$B@9?T(u`hZb($q1*Dqzhx8l z+`(-PmfKEWa?(sV=^3DWjebafu7fyqllz9}+?k=mIt)fiG~#hn%zIPcbvIa^X!ZMV*_fx!NP{xI%kmCanT|R4AhI`T)Gx{4+@gNowqLC}z63 z!_oPwaMj=e3v-UqrTu!3g6^{jcZGSHFouiA2FGrFad(vCnHHYcZ9H$ksic!=wvlf> z(o8Tx^3X@;q%gnq12X5_gYEVD4Q_6hJ;W_rdLb)Kz^8FGYhibWR|=cffGmspuxIL5 zbU0HefA;RcagWl@vCp0tKQC_R_r&T_84wj;7X3Gd$G}AQBpsO9CV9W9irS~OoYTnB zm|Pm}W{)P)Z@ieAmYHqI!hwnskx~AVA{s#WvGrKdNV8Q)YS+k8_T)A*lP{*_a!%@Y zC$<9t*XeYN~`hA6ioYoH*zb;*s!gHAVJ zTHjSlc;fwC+m*B`S{Rw~A3HUT^9luOE9?y-o>Zg0h*2QM+FnoTiNqmHK`p`yl zk;Scw-sknNhk^(ivLBz9AN}yn-6G2&`mWw5%i#UdD%t0Af@V|ILtYptZtE3>m7o=6 zr7s=1_0~K5+pZqd_dmK8z6f39=v%bSqryRIj`v?Esa8TeC-nAWX|LKw80@jpjx^vI zBy2*5I!v<9MlrBlKCCUmkg>y_*OF}K1FZ2E(j}%>V z$tNXThnLvhb|jY{zUzEufyMc|n9_PG!Pd&U2?m4xi_KjvU>-HWdyMO5zTrAAvXcL4 zsDM0go74Ia`<`!f@C@!TD|6|txhgKUeK_6d=r)b2vPG=__HAWfbT={tWh8YT)T>IE zNh#;afBa#*c3!(|gxC7h?A_A7B9p%EJH$Ji3eWO3)g5`jKJq@qR;Ot$y-!rIXURBR zhBW366IZhtJT$}kj6} z+!1&@i`4E%6XG^Pt#_Yp$UJH{rIm2^C$b0ceuMzvIPvpMe7}(^3Xt{Cr z@#{(TH0DDWHn7{f+`Ld(T%P&vjZ(yCH!XkdQxD?h6+~05?-3(j-YXEc9=9dbzVs5S zy}Iw&tjLQzgze|)&3ZFYx2Q;migS(v8vuW_bXv)seqifeW3GZ$92o}tRMq(p$MnyO zG4uD;Lip8Los4eO*zWUlB;QQBB8y==zFT1S_BXV1%R@`J#1tKCN?hgLl5asB0u}QU z51p!ui);}J+qimD$17FELKW&d(=K1e{2<$l%!tl$FG)#FdOOm{Rm`e>H5K{rAZf(p zZEj(bl=MO21CRGb*>Ua}RgpV~HXOS~XZ_+@My2xOxn06R3LmKqmWnft`-`z+@R+^e zwMXqc~0lQ}FOcfdj zcFF(b3u~q5`P0LhH5blCh=naqD5dQop{_3B+4^1ZL0RQLqV|ZDamh^b97Z1P`q<}k z`OP7sh2mBQ9<80(UFK7@w3Uixw{i}~9hi0|O*GzDWbD}5Z$~`u;#YhsaBrZLul_*B zCzG+SL00KUce9gVkE#L$J)d`b-Q<_hK7C=9eIX8RBw@C@;~jkDJ(EPj(TqxI-!q<{ z@9H7h>3^tmXHUedPP2yJ?lYU8;F=j951I^j@Hxz>GqUMUiYUKOM9>-ArI$lruPyn- zC=Of_s1}@ic_@1*sB^zlDsKQy{1eIY$%As%!vYFV(=eSAW#41^uR3HkN9xag-8D4u z?I)eEr0X{-5~Sv?;QvCqw2~FM=tNmHju;k-eG#Km8E{2Q;un0JA9A3K&ZXxErw`Nbi;uVXUn!0|LbUvvXJ|;b5 z?&x8q*R%1;nPykjjZe&ok1LgTiNF*-QyD82PX+@v^8TncIFn|t4kYUgj$tzreoVY? z9Ilgc<@jmE$LDqKw}_bAqf2e&0=uNF96y$71fDDlJ?mPw$o?sLJX?4~w8?BJ^9Vm* zX58UDLbbUv3%)G))(yE_0W2RDnBPl>nY3vZcz;;nXtLr`-NLiTGnHGJY3W4E79V|K zg!iSzn51Q6<(HuCSD7~YqpYUP*tJedUTF&zkQqs_&YSkPf-QL;rQPBfEk9Equlf9N z8=e04pDtmZ11dH%Z{KR+gJR_042H#ro$lLCqh=;wsnY29{Xp~U$()@M^2DKwMW|Lb z!`FJIPxoEz94JtG`URupb|}%9J=d6devB3#JJzNhrE+PYO^)ueZu24MJzw)&BwOk6 zu+QaYLf!A3W_m4r9xZlGb-S8)CTD!Q`-Z*yNddQ$u#ZcjP2J`11djTnOEmihj<`?} z1r?VF4K`!{C<^A`Kzm{;B|cyTE$L|Tpb!XWI!(YV(<3u6y6>ou@h?!;p(7^#Y_LihFXxxP}| zO$ASGV!p9?k?AaAg==l#gC!2bm*M`M9Q>^9Io$EvSoat(E>-&{<=e+^Rk`^=gC`}G z{WWgM^2f&jwu@#-5-Hw3tuO8K&a=~T3Z~|^h}l`z%5iae;M5ET+N@>6ny=XZ+}a@4 z#H#u^?v`J!X=35^gTAE$h*uJ3kJhsY?7FWra(`p?wN5odd&rtc${gsA?f9%>l78dq zb+%37-i;Dt%C>Ma~$bs zXR2PtOw-V+Z*iqU8Wm>*4PKM{QF;c(U0y(X-UHv)uJw z7Jf{($?PUAh!K6dnqFzd8fJKW^d9WHSo^b>l{RR}c<-hQ;O{+8WfQ8ANi(iL}b5-WF7qhI|dI6w7mQ@ZQE4x&*K-S3WVr63xe;|6+1Jq zqr!DN1>>?BL>93LsxpzeVj=1sNEu(eZg{Mkh3-BTn>>BdOAPipUxs*;wi>uZob5az z)R@*aMK@<^QyVx%^KO&*cDs63=h_QRzRHgDxQ-Cs*PoEiajCoW&Gn{f7`VUO`+9uV zdoJiGjY9{L@`B}eBovN{wK)WZgbr!kis!9qVcP^V;cvLKecbWr>p18X+2 zOJamCGUb$pzQatm^9%94Mi*4Ji(P5bC2l%zOrSS%*Js7#mFPXsc+*N!x)Myh@POe= z@|`g&FRZ3OpHfWmxfx!=Z-{`j@(ckI`<*JYF~d9E5xm)(ANW5TXuRNG$Sr(X`eu=0 z#|i(_0v=y}%#{%A_CE-Epkl$s+c z1@znLT?a%pUZ?MRYkH&`mUjJ{jLplkk8@quw!Yf)p;kQL#KZJ~G)||uQ|I{K3S5d1 z*v4F|z8g2W9Zsv?x6_e3YM|aB?E0`%kk_t<(Lp6y)tS@5XjNCqsv5o8e&d_%h1b;nbMt-YH7R*vzvq65Xyc{!Z8BPJxrqo>QM`;d>|8M%e}WpfS^o*=L94ZCvM-UtW9PH?V*z_pMYdmJ69~JtPvVaa^;y@c_$*ws*V(eTSpD za}v0P?tb`2e^Q;1_eJb@GZB^^bN&Nku=r=kiL!>2Z&R>W6Ma_2THhKIUSMe)_PKaO zyz1m*a8HlG{!F}KzPQN5`R3wnD&nQ$%>Iq`V(<;idr3^G+A*q`DRJs2MK|kgxYheA zV&m&DD@N5VMemW`jeG>3>9FwZ71I`_)h z9y(ulW`m$UIVbLvWeZ#9eNk36`&`B<-=TzTcQ?pN5M*eD}Zm zNayu$ONrRvoPuoEmfU|f!oTB+j7NC+_#uzEo~Kz}FWp5#(}(YwB?G_dJ87<`VIqG7 zf19HhPu?uUvtLi#kz~xmZ{8Pn(6n|;u-wYa`u$C%&9^L^#3u9&d2X;~o^+Vsrf4iM zv{yaZbm2imEOls15_WKvgzA|s`I{i;fUS*I2 zI{s*sS$hAeaV;yguejGzM*I`ozac9>?0TV?LVkOYSd8hC*={b!SYOUMt}h(U-@*1& z8MZK`buLQB=Ut8cxzkq9F8cYm6s4&vyGo3|9sgXmJ1=z@?X?Xvl3(rHv*XFco6{F# zZQh>GIWwKZoMLEuD(T}Q-NjG;Fm1Q>cc#L3D&8~-{}+>)Rx*(bY;@8GT#JybcbZZRtGI(C(r)v{=C=K7;NM9(R>Dwc3SEg9oQ%_M~dI^Q8(IpL0H;xKl@8$2nE! zvT&qWu4TPlcFta@deavT9vwHXk61svqnY5Fuo(A-aqpthTif*a29Xz2QQ!FYwkSrP z(*0tu^~4(W?x!ZQR^anSAFUk>QE|2BM_LCQVtN-~!}6W~u<1t-rl|~+im!?R7>YkC z7zGdxXB`cI*%e^EFij}EAvZy=$J8_}ZFYxw)T@}dJ)u_cE5$>%t|auTM!&dN*&98e zGkQniL{D;jW2$WQ5lxv1)Gei35p$IvgoVFW^>)=W-A0TY;*kFQAc`b9-XZYVk+;*4 zcfztJP_*Jmuvy8?^V=Vd$1kaL1jy3MhH4Ln?VrfMnE3o_Dxb$fw%7gYkQ#l(ijo>Z zPon_)9s8}_t||x*Fm-*de#B|oOv@8}71JN^V&{De?FPOeyv_kzPoKmemU{-Hy=}Em ziY4ct-=v}V!*%X!FlGnS~tDHL1OjUYnfK=p)_m8t56p#I+kqHP5ST7c!YU};uznF|Mc>F%FNkrH;h{Iydk!Z9r z5{KOfw*Jxdrf^rV1A%1fMPJ4;)~VJp^J(Jgdhu9v_*w??1Z`<$=W^3gp@y2{`4_7#%G4HilScy}l!ac?QHiPmTLtT+fC5mm zKMqtYNspag6l}%h6(zQHicI>U-$Vy2y$d_z&F!oly*K=7M4__WLAo=%BBBD9KU_JV znW$x@vNy!kf}`tsY}M@^(I*8m?;~`jVnmJOv$v$klDw~kjpEtr>N3nq6Kmp&kx8E$ zJb8up7ujqJOx+cEOA7I1yh-Rz=}`02qWIS1B{ZmwRQN>2!$bd#4rbb~+&myX zALV-PnRAwO?8MSAb2k$sTf8dS`enhXe7MZMmRBj2?l+%dP!Z1`Ha~DhaV)8+ z!F9iKQCr*$b4Lx$zOWxno#A>nzS`fP$dVFw=8pXs{&t%wUa@e~rM)x$6({S*PD}gp z&M)58n0?1^NlfV(m0?qH;Q#>ckIw<-64n%$Bm4%gGFNR_*^bwgU1JEVPjYvGxxVrh zu*#LRwA>)D!}YbXXm*UCoU>U#0gbBG?J|cg6}y(b%8X>LIVvPk@r>mEVwH|FxXKxZ zgq+IVZAm&U9GxanKWt{Au$hMWoZUJYKH=aqbZ-{o;DHp7{&!d?JOLc8w(p!l)2P#f3lvDC8fRhBG0ugU!d81>T>k z+fN~ajf!^m{k-0H>aK-wyKPKEx@gWv)3kEYS7s_9IdXYTPlkIBU?(#hWCOw)FSj5r zq_)Mmo``Js$Zn*`Rie=tzFTghu~~TVL8;l!nkV!sC$D&Oj5%Kl&$=u+83%XTDIlZP zcV3H|&uc)U1a}?m{XETjH+ILY^mZnHI3taPyx{QE@cRs9qaAn2yyyp&xE}tE+*;d| zz9yDEGc!0d}`Tr2=0}~TXi(laK5cB>;$^&H0FG%@i*!C}wZ-9EGpdEil%EMQX^6Q#j3;PCm zNLehH08j*gfkmN#)dv802#o)~gXCeqfaKATih`KnRYd&XfaJl3ZGf+%GwFA@JeXU3 zp}1pDk}G*7>rcU$1Jp3W{jje)z{*1~&t*V8Y#pE;wCW1tl)-pMPXNt@Bm4qM0b~zQ43Mn|Lje4SBlf|G;J+gr;RHuG z!x12g0l@cN;Rq1n?G8tH0MIVDd=QTCgd@CQ2tXfz`1^vE;0QoR06_Y~5rJ?-5a2Bs zUGVP+e*iUJgE{@x?*6ANzli%6g#S9(0>S+c;C`x7{=Wh4#{ew&e*p`W@j-Z}{|CVR zU}G8!f<3Un<)mPbF?bjeg31G=H5%A>3=x3NL0ScFW5E3wJPJ-Z0(d~UAuKsSSOZTI55R>u2yPE00C+|daWIeoWF5foAy_`p zNdf>{6Cm(31mA~_&~<(lmURHqgWnT{*FY~Io%khD^a0X| z)kM*i6`g?e;nxJwML7p;K)SJ-ppoN|I3U9n9f0&;O(KHW2Z~;Rxf}mWTGffwBg7OS zcNpae<$p;OSubCQ?)f)CdI3olI{pjlr%W5GU`Rta1#WK&*;@+69poHA(9QDP<^<%n zoO1?|OUpSIpkd27Fo{s?C@|!d$pUmlnGu)GIuH%z%oUnymUGbL064XL0?ilTwFl+I z4M-Kr!GLK8s2NJx%CrOYl5*++!Zwz32jKu$O)2vPF*M6LFDT@TQsxapU!WYA5}?2? zATUa?4~UFe$$?VJ>me{wU!cm%rTt!>@}@Z4iXB(ttJ!PH~xj{XPiP%c`cR zp?>u*{{2h}LPHJQgm88Qi)hdXLJj>t-UeP~tlaj?R0Z|E(#3xyLL*sM2n}joA;g_k z3YX{Y<>(vTuL_0^#-VH8~%DH%S-*L@T-kI)m?|{94V|#TD-4=zI`#v)Tl> zmC^xlaWtWi%k0!tKm(i&5|~F63WLJpq>=IxNTh`1FPsVXC%MAFM2Uof@U)d55H}7y zX0Gsm%b-X}=m+;+DT5qJAh$JT1n5y)TLvu8+A?{0z_2y<5kV;4x-uwi=ePU738af_ z%Ak2=O&JOcS)eudp^-pMe=kEp9|BlY1}49?^dT_>0| zQ8+C0<&EFV2j&pKU`$E=r|JZNxjT_O}(x~4rM3YrF?-+wY9 zq5*JiZ5b8aCE}pDXH8vs5V7`q843xW^mS$Hc>uhxb#eooU&j{|3JqEO-}*(Nu)rf- zTSi#lFB-K@zF-XsO$EQVhsLjaj!>BO&jJdAUMF9G+g|rOflOic@!Q46BpMq(uVyXt85&0 literal 0 HcmV?d00001 diff --git a/analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf b/analysis/pd_sep_paper_section/figures/fig_kv_memory_wall.pdf new file mode 100644 index 0000000000000000000000000000000000000000..323ca7c2e265553ae782e184696860b34bec2464 GIT binary patch literal 23568 zcmbun2Rzo__W*A1$PA%JNQ7rU_TGET&K_m2q{tp+Z<3I`MY1!pcVv`ND1=B^QU32o z_4)Rx=lB2g`v3L1z3+41cbs$Xx#ygF?tLE)4H-FJC?A4^qjUyT(m(c?9*XG?8s50DWs ztc)B0inX@~2(0J`fFSkbDgEQA2r~Hv9O$12py5VC?`91^_Z@t&mbJUHr<)~!4;p@8 zd{t{JdvhsgZ(u|S@DGJV@bM!ApeO;900aqw!ytSD{BRTu2@D372G9ihat8^12UgO_ z$r<~omF(N=JRFqC zQE*f7gBwq$vaaHSMVQ@d{4JbW_@{ldgZ_`B+iL~t;uz0hmcsObxy0nyYs>k#AV?pOacjuzIrh8k3s0okCXb=~x zw;$O3elJ&+sGayqY|XV!&indnk_E6nBi^zmzCA;e+xM$SYe!ci&$=PRQkFcmQ0Z;& zOXqlAh`%WEIVz$RF8g9kOjn0Z;u9`=kIJ);MZ%}2bm!t$dG63Xy_xhU`!*kwdJ5zEIsAFL zXM%T2hmr1w5%ZG8SkUJYqscqVLboWA8a?QJQNdb1-yA!undi#g3dmC=g|AF(SI`b$ z_b%x(5n|fL+MPI@YmgIp^=5pZ)rf zcG%0iv5#43-P0`lG5qo6`#(ohwd$a2+Uyv;^8>zG^AXvU9T&vHG`aI-0Gc;L^i3wh z>2^QyJwp?mG4>CAo||um#ruA+4VDvnHS0)nebr$%D8g>A*)__$wh;h-2Ha zs9|5Gmax9LH#>(My{9_63m^6hUA$u^@{TzsvH)%&nn`rKU^i1#Ylwo}_hvtTjE)tn zpyu4GxP^yZ4Lm)7-dpY)mA>{P3q;d6=AZl z;-ez{$Q9QMC4#p_=C4>cIeKlW`pudLYJW%Q3AWv9=9K3IcpX6s;%p8!Usdrq4{qTZ z-RBLwKU78}#BXN8S}U{pLbo!Y#4DvLU)QwPjVjRpu^sXL%e1w5NWU0I&CX z7*_TRF9>SRzcOw(v~p}(Vrp%M#Jo4i{od(Y^fGp7AT1?+%8aE?ln$P?L|#HqSnclH zz)<%#Fi z5kZ6ZNTzpn_I+3lcV!6P9*F<(#!co%)hwDnThcXAtpzA4pr=y4`A_NQs-;j23sWVl(=Mpbi5YX6C%MM)V z!@oPV${vIDC`K!jREvBxT;EH+9?voHiT49z8jcl4n{weyQdG)J=Wr$&Z{&v! z$nX^{2A++tSUZGF`Crq0;>}RTZ+g)4I^%AIg$+%qE8j(?{`|xSi+07(wb(>(kY!Kp zi*=?@jd=3|GXfKpa1aOh?8ssz@6=z^ zqD^7N^^zov>-}8KeSt{U$WJnkLa6rvHy>`YP*#{y;807tC`)@tU>E({$}T#|XTpkt zX9G1O&&os;>Oi-aEG%^J!1N5`#i`fG6a!L8MPAKb<)`EtTFXeBksRV^;Es|ef$rx% z!i%>m48;=-)ux`x!? z5)tybWr>zFe@>3rq8&*+QH4~lOKIE2IE>eK{4a@FFo{a)5UiCjEqr!}HQ+Z2D9Ejp zY&pNWXL;_8C%4B(HQxgL3*5^2?aqeHuZ2T>>%t(6ks-KxpM#QxiHRf}^FT!|E;*Il z;Vm}x*zWhZs+Cmqws~|c?gY8Qke3N?rtV~gm}Lcr!;jcmukXBX4Q4KU$zzdm7dfQC zaZainPez+dR{^4Jb&D(MwuO?l)g_CL>v(j1(9v-s;h>HQ#?m0aEXg-wZy?^$2-Ztq z_n9oVbd3puj{-;3Bc+--H`pE~>6Im>K<=gZ(pJ%ZA--_-$_;|&tlqh`IW0=rh)biv zGMRw{OF?@LSN-Yk+>OnTRd4k&@u>~E)dRhncL&#Vm83-DPK1cc&U+QwYp~gp9v{QQ z(S`{8hrxrVKlfdS=?Vly;^oJz=^HczV_(3!P7BXvR2h_~$d6Qh?pKC5lW^0CUBr68 z<09cw^Ool<+%ca^4v)5B+ph3PumS#=ri8R@{tjU02WZ)}&02l5KEE--@f zsk<^&qUhx~i+WHFSF^V9B%^m#Jg#PlbRqH;r3-G!GirwvEo%Evp6c(hX=2lP$9P8Z z&Lg=(yr7x4=}4(08M9>78&Mjh9hF0_4*1mBhTL5QoZcS6+%e_SnM3z7HOgMg2URf4 zp60%<5^eFB=$lL-ENCP_n|C-=!9YC{&cTACnAf;b8V>kshC^#!kF-gK|?JeRanESr>*tebn0 zImokGF7?|*Y*(&HT=NKGF9=VVu&nNiz%F=HLd|T=z_swcBLi1OrM#PJM*#&R;aA>3fRSd}Ct13D%zshVDT(dX>c0r^ACkvH}c9~7DF z-z$2Vq$S_DL$5G!mV#ix6?=I(0r#1ciI)h=h%+*G6dkj@IKEriRQ4OH=dI%C9=S+AIqq zY8f=PHJ%-xAQvWW=9;C_cCtLOvxbSqm$)*T%MSV9k=ydoLOzRlD!gJb*B$6_$Ic=jFpB*W9!w z9mjQvCuOJ3H;|z1Y{FZPb4BOmdajBZklhY=6c5`8=`A%pwv2bzP;3|WH`<> zVo9STaD{E`4fCrn7?C@ASrkVG*Lc&LZ4?zM9%-oTv_zuHW_g{TQFM)8Y&v7zIGXk{ zd{gXkf_vWTvk$)Wp!C~kZtpfP&$}uVmBjdYirK$ENXzikGq^dPN>Kj-=Tp0Wo+w=| z$L?}qMRLW^+)@4&+-onue9thD0{lz@bb$hztSYs(j+40Tcnn;N%fltN(vy=^`FOK@9Vtm^W_CXJE*1=b z`$%jTj_o{eTv4+7QeIaL{NnD_P1(EM*8~?SPk-K*7A>~2gtdqjeKa-i_|~?)+~GX$ z=kD!Ob=%wT+q?D2w`)(rIN!fk@x{-L5#M2ZTQupT@7{^ z?c~1gxx*k@@^4;Db9*xuRg_GrGY3r0>^WEB*+P2=er#xe_zF&Tzvvw?oL>O;(}8fZZAKRwflU4>LwgN=+d9Kg2w=bbANyue zt@=1zC|Um#lH}GaTB0rWu4a1=SXxDep-^Q?EXh45O~JF`jvW0lM^o0eic86RS&rKc zIqwOI?t$EUU3+OQlDF%q*y(t!FHG1L2W`8ax=P!!MGdDOV^w@c<)#&LzBHyHXmW>X z<_iDn5hKGn4+jk&<@k~;osJf_8YLHYNdzlV>NulwhH3@shcRlCi+^OTLJ zJ$Ft)mXMY1=~zGWBc5EQ@mKYJxNl500`=9)MAlE+&*}Ln(+<2Fz*Avx6fX6DY+%{0 zL%c;d$U1HqK5#~;Ec{?;X0#2VdxPs#U7v8(4TVG0h9B&_E zx!Bn@V4(c3Zkivy2pAms-vmW!Dmr07i5iz_ze(g(a&D#-zt8@9{5Vt-jEg%p1#o~I zyV$eX4;L@r>U)pq-x4-h>Ks;jv(y|;lFQvrVox~irUxg>Je!7-0{>`WBbG)dOsApx z%++fN5hNLIZ!F)x3K-syrB_Y(~{?#xm|S7i2=D7yM_s^lmF(g zm@u}DQks=WW>c?0vLRh)#@XKm@;WTe;y=5dyM^Sa5p1xe-UQJIs?mXa^*Q;??|Uj}TK#fY~f=N@h` zuCquHR$cUPvNj}5(-*mxAw2MKVA)UY)pb$&vi$KHtysyWoj26)NK$MZS8y=XY_y$b zQ#M}s@Jxozv{qwMDr~Vs&Xo9Gz-G`y#y(@!g7{0h+$3=wBM>!Ilifp;pK(utzlmhj zFE{XV@t*!C%IlW~jfA|ap6<2Pyo=*5krG*&VwzSkoRk^;C|Wb;F743=3J~LGC6uv` zHVw&ry4&IDh-iF_s^%;XlFtv`R;*@BALFv=aVYF|b7tV$V>!w(|)K(^*iHo(-!xmn2qobZc#J|DJtWUaYI+c|>R4ylqqW zviC7olDhE)4D|ce@AzYZ@eBSp`QZFWEr>>C4HlJ6{soeEa>*oIp~ezo;Yt10cJ|~p zZ~YA}bP-4-7Zq_)k)9_VBQl0)QBw|QrfcnE@(ODOwW_M zvO!#m-b@8jVctT!^~QeokC9##jeH)310;9F*cQ^BKtAo`&$Lgm&J%9lSv^)AYAOs=*4TNZP|EX38 zk@TiY8_^qWoI@R5ir#ONp|^b6D2#gQi0R1AJ$xCGgzFqw#p!8gVv0*EMaN*Z^h)^5 zT_qd3r|B1psGqaD&dFp8t18{6s2O|pI%FAlXO(#!1B7S~`EL;N|HZ;YMzx_K#61Qe z6bRASEh-j!{`D!Z%&m1e(Yntgms}OS(ak3;(X%KtZXi&duT0Fg=WWk|_m9s7T(8CwmUxoy;+<`G0= z*$pDwP$Ck9?Wa})`L+^n<3_Iqm4;czd3BoxhcKARmVK;#d^*c`ljqp zr8Z#|x^QSPe;O5|8TWNJik+PTfsI%3u@0BbNA5G`Rzn8jjzmvuyN|es#~ra?}GJS2!`oO1cdCBB+R*8HpO-lcG+=|AN#h;a5W)#_E}u2 zmLG}t?3Yng#GTL^A|J=>s6-)YN)Qxlb25PeVRUIod0-$YetRQ$ABaT zJAohe-#EgnA)3IN#Ik_9!|~nOltXBw`PfL|SrU9H`1+u*{*jT>eJ~VM3xYu4KT;Wt z%C>++Nf;={N(yRx6*Oirw#V{-QDtL*2ZIiVL;iyMMdOv707FoEPb`G*bnBPt%ul%& z8&c0*)3a1G^vnFjJSqm#Qw^IANx@vnGG-LH0&19!{9Au@r+~Z!nztS;8 zdM(+3&RgjVKFh<|3(Dkr)2|BEHp@h&NA}JcPI{dCllJgmO@i-dfjb36pkr3A^JS3Le~S>w72UCA8NlKD2OX zr3t>gKjJ&ye_+x`zsXoYk;z%yj_f^aKRFXQBdAP!zwP#FdIWSb;#S*z&P$8EHHGjJ z(@Jl+XMg8G;N<#EzZk~3wSp9fr&AkeUikTp?vmDAkvZh5eE>Vae+hp7MfUL(okLj@ zacd#n3bv>Eli<1C4hQkhUEwP$XPgf^Tl}hCf**ZtuQ<2^_R9+6cc`THVRbywsxhB#XbA+U--ci=7(>`Bo#1V zn0^cv4Gb>u-?Bh7uHRLLl?-oFCj0crnegR$Pk{#AB=S?WeiVVPlkh zZ(ndbZ;|cxyppP)+HGK9B-9*lruG{X}RjtwPEkK8Hzp?U-Vg2NPT|OMXJa| zAYCI{x-r_6)_0A>u5$jP%>DB*cdoB|eFrC7&eYd%l~Hdbl|Ou&loNS1?YLuMpoJ?< zMR6rNXK72sg1UUjIl5ZdJwc?KueY7KEWj_MFOK3XHMZM0+xWl>wZf^8in7d!tRQh_YmiYbb;!Q@uIX*3_R3!G;wxjh3y_sYW2|&NpRHGaadTg;eet0g5fj!@yLf4{A}GWAMvP%$hVihR-_3? zd1LAo*nhUmTAUnyt|>)tD7PzttVb3lDnbdH;y5h?X(aHRvUwTN!uqC~O>0mtd2EvE z^K)*zKJD81)j%r8k}vLurKFCBFR9hMZoXH1OC7$PT-s{ANSn~9PavvUuMG8#eeaOX zncZ~pL6C#k)o_`Vi?**L+S)FTkgwNo*e>X@+L_E2I9e9UDRqxMJ*BD|PjnzdY+H-? zK=k5Ex7Yf`Dl%K$iekC4{@z zn*|D*B}ZV#niH|NF}l^68M?O$fgWj;eb?2lIaIb7V8LLh!r*^l&?D6aZw~rHf9P(Pxgml7{sOH&Od{3`|c?wW~~ArnBXHg>Pw}@l3GSDbj?xD$290 zKwDoPl1&!uQQMISa@WcI+#3yhw;~Y7nR$0}_Z5$qpda}fDJqJUc6p!MClD2svwmX} zhtW;)*kpE}MYeH;19X;{ZbH-IL&n?SE6rXWERQczxl5%>@=>!6u5#y}PS+@{ewd}D z=~5(Cuu}Tq_S-AQycp<(!G?w*{$kH4sv#T=2-r2j5bzfJHk%8JTnVqu=6pGBI+uou zUoD({gfc*$44PQ6$WfM69`N+8)JLa! zgTy{@$G$`oA)8S=9;*7+R@ABj{f&xRkqn8j_bmBd=Y2~>dL9a`Sg8(`vZYOrslQ6w zXL5OR<)xKcQnv@oz>*P!`@;9icmG`<;waVwLLdjtC898zMm&m(k%TlJynYB zZH9TP;{(EPVTR{1P!NMj4j23jua>BmvQwu7(W_+|CQGCI$x94D&lU6 zfObKA`(o?gDw|NbhV(u$EjJR$zr|-{n06z)@op4{SnW*82i95>556h^X8CPW*nMu9 z!olLv?0u=^5W-lst!&bEU%_vSVE?ws%2Rc3L}8MZ?C-dwx|}o_(y1gV&6ne73v)Fd z7_`r8epuUs;a_!*%psrY0q2q?TvTe-{CalO zzj?ee9Q15*Bg&C@t17t#T#S*cdiD$^c0Y>CCx@q!F}+C+-e!K9a$>Ke2bb}yThnPo z!L{=>`seRGwvdUs~S9QxLm)Zy7 z@Rb}#40DQU2l5x{vl>#1mS_lwHUz=11>r|&L7+&qM8hEtNUb-9v@t>Y=?i%XdG4A? zTESZ#*tE3^Sx%GYbe}G#Y76YKammBR(rjU%!vGP6JqQ&37aKp-7)f$mC=u@FjgSWt zAPMhbJza`d$5)i6yv9gbdi@nJUCg+m#bv>os_yatqRkY= zb$#11#@{vZDo=r(x2f`wYR4Jg>(sn?Whn^^&UGJB_T-Zs<4(yA^O8zUI>aBq0tj!> zr^{^cS)?&YGj^FBS-I9 z@xbfqcN1Rx(!ATR3w=L+vC~U23mH}3(f7U*B9>Z|=Jg?7%IA%$@c982l3bWD@1@5# zpS^bq_#)n~we_`kZ=bn>V(`cqgOP{8U>gJbo&P+Zkd11iYy}bF9qULElaRbrzmNb} zDN(0kx6GEwKLq2G*4~giho5x8lvbZm$#O;`MxyzR8*lFnK7|YC?Uy4u8OdYU?c)@6 zxn^>yZXA=_UWhHcF=}~#kJR#zO$5U{V=zVG0OCLkVVphT~pYP$p|yh(}uR8x%J zI~(0oc`^8!*v!GZUpwEbud<$ljeRmv-S@n7Y5ozyjs1N7z9KQcS5i#(HRaN%^|(Ts z2QNP^R(+PspQbh3`7oL{n4>w^*CW_-Iy;lL;nB@;^64!fV}*vz#6ec3{$q^*9_v{* z5)MhDk?ZS}^AybGftLc!E3fedZCJy`kH3V`8E^XTS#l(rvh_*#2}#?j8+GUFB?(OfiKCtu zUTt&GUIiDrcLdSRE+U<2F3JL3I1+c77wCPqIHul#FPDd>ZxMSoSb>|`Ghe=2mP#PH zUWP?(YSmFz@~|*@V^t(@*H+d|E^_>?fDmiE;d8;jh38o;hKt7hRSS-6RrQzWKb(7$ z0XE)UJEOcF{0zhFV6b(ee{;Mb8{6%Q54biqi*Yz0{v}u}O5%Yu>e(JV7!-uXfFw3SF`XJ5~T(Y=`lF{5RUY!iZpO;=(lt0Wd1`D01 z8i-#kkzl(e^r$n8OvH}|rWIW}MM2Z;l%b05cJrZwNEXUA|~y2f(# z<>g=#N~Kp~d<8JI*=AhBH_b^UqAxeaS-ga{F(8Yon& zTr7@PbiEj;fx+?>-*aWU>wT;T86{vT@yTKp2x@36 zdt=tu;6c5qz4)RlgNnG&TS+y!fz)Tbre(av?senZJLMM<;)BCy?o%`}Gt6QD4TDMb zH^)G5Or%Oat;cspg@4d?I}(S!(!0-FyvosWMt!%-S9wwb2T%N z&HrdgBtgjHDrbIN-6vrti+a2#j{Lv1LE7eJ!n(>dxfw zy3etLo>T5#*$48y2#`9bwtVLXXnKo~Gs-uqiC7Z<&j1I(OnP%2EH?ulciXAbnbZ+yHkanOA18Y(>en4z3$bK!dGgje?!i+Eah z?7K5uh0E9Y3?~_c<`UrDOZkVPL-l4UjX^4#2jUaMU%uh8aM^spKuS#J3gD6ZZ>b+E zOohuzCVf*an{A&BS8Uh@#)aB*ww!X%Y8}%D1vOF!j=v0ySP2ZP<(S3aEW@+4JX%FL zA*q(|JpJ`@7wrylRQ4!q1-Q#mQgd?kozP)WXL)#L_4>Uv*eclVdy~GW#`Jy zQ;S)_dl3P>eau9I@3m(RP{KJ44ZYuTyJt!#{VuKRn%f&F_Y*(4-)JL2Jfki6vP4w; z9Es5GpSU71STbqz`F5Rf0o;37yN?#bk7(QiUtEJ2}YVDnw9jDWPJ;qQ+GqkrfC=SX1Vn1O7^|j>j{b7pav`2LjEA#qBE3OLrH!FXKVKJli_wQvvSbPXFviqQrGXc&wU82m4M5vm%!j==04Hj*GA ztt(88EQEJO9(}zn-A5dsVyC>2y`5;3+-JSVCg|MT@K!oZEy;e{X=JdX>f1vy9Is;y zozrC;RZ0WK&z-6eis0!EfvA*gkE551ZU!4X^xj&W8siFB6@|6gR5uGkN<@dqW7X=q zD&CI?C$Zy3G{k%JQATl$m-D>rXNV~KXoMa_(M_n(QBd(2xn zdNoOa0z$1ln&es%7m!FzAZ;su89|!Y)8`|FY>yJmPHXOPF~jYz4$jRh(`ury#$LfO zc`3ArPzsF3YM;Ltx$}jf_{fwW1Klv#HBdk*{im=MRTl&f1<780CGi9Kd+OKDC7fT0 z{dT)-C5su4GRwQ?(L+moa%g~JH&aAP&AB5)lmt&uI-8GFCxpiZr5F$?YM@mQ`g9E~4gm$W9;HcRLWv zhO+Vt|pHtU3(+`js^;e^CsI6;Ax{-5X}WBN}4=t*y;2a@8A z3_lW4zmpZ_aeR%V0G})s(#ttb7ZevMtzx#GYWapkN4-ppo$X13qTuNw4SqazI~6i` zMy~Sfq}3K{ktg1QW#jnwV|!)|91$`MgCe013)X4XzJT2m3X>SD$$Lt*XVrT80%=pv zjJwSb*Oj?tQ?t}w>c|o940pT7V88!$Gne1w%8WPU1%~OvV2B|gf3dYkYAHWr19nJz zTy?EFX)#bS^?cK2*>~Sw#Z}&n%fe&Fvl(WUewQfzUh8HbNy5`FJVp!o+nc@3XWuG) ztm1GFe~~zxKxq*XSwR1u;c+0td7>&QdE_!X2uEe`qB(W&aJ8BL(@6_2NAVXSUb!jd z$!p#)2^+4`O695{jgGc#cE_hdw=(D9I)b5wC5%ZHbs2Wuto%u5X!fQ<+Qb$&6s>x) zESQxdcX2>EOXSl*an+e+`NG?RM~TC4bB7JeCaQKKcw!~7nPXi=Gz0jG6ec&BGJ3P( zkdTD-b8&$WB2-l!;QDz1@`1`BL!~0nsp?TILRwcrR{;%Tw=Bk>SG&;v@E1 zxC)9r(lvglE9~_^t`j$XH?au~CpDF|_C-fj7sl`BHvO+?N%S zw@ZTt4`sj9MZG~_n_afD4}9BAhi{h{i1-%Wu6uMuNV!_kg<*Oz7_)!Fy+6F;2nb=k zT&P5HT!hpe)-wuN&j#KG62J2|Adq0p5g;tT5Oyi((kZMrHK$TB42{8{75IyVBO6tP zmi2gOSx*LqXwc+=4xeBf!3K{NIlndvP?0}Zp_a4mxY?Rlvt`8YZXvOH0j0+MN@m)^ z&*vVa^Gj0ed+)i1Y+Ff(DYC_x4KM(P!90W^{=z&|)od&SPSn>#L+Av61ND|0eBR!j z*2k#a47?&4X&S}e4($f3K676sIfZCG3P(m|hXxUTDebU3fkoFM-Y1r%jpa8sN(%|KoA0Pw62z9SUu}|_ zro&yW^`^$Yv;FC`;fo%bD3_?Ch*bieBlY*jiCd~c74gt7^g2z#L6M64rm{1J(2Z|0 zkSc~e)6oWAbhyTTXomWfQtrJA{%ALd{ z3%*adB(mz!;0@e$iBFJ~BRBgDeH$8~bjtwrkOZt_bti?CI zwR1a)*ryin9*cFl^WyQ|k{j{Avh<)L`t?CPo&8>_<5;;*rHXKIZY7h0y1VIxD~7i9 zLM+3Cy}RX8lv<5AG$HlyA@?^I$MocC=)6!0SB)KBrCUf z-~!1)Oa_Y^H?H~n^l}fkv%0PYnFGjscb~u&ZPr>8s?yqTmHl60fbw+02bAY9SDi%Omio}A z;^Mn4!08KhGp!25Y>^9W@LaiNoa0qq$CA;xJ&r1=xtNXg0jGK~`F-l1zNcns1fV=g`)8 z5ixmaum3tK!Z8Q`!ZJ%9$G|fhp~`NYCY$sZx+`YwEMZ)cmr1J&Ssb>FsFQM;%N7}? zA#?o@ZHw$TmH6ZDEUW`Ryj?_Aw1;nKCYW@%d`KR=PBd!w?JQm2YfIXAfBAwcs#Ep? zt>LYirUIR&pQr`NTf;5yr}L0m+FdB{iH>L;$*j&Z@9JhT-1$VIvwABG!$e>(LjTUW zhPn*Fk}^?d!?y&zn(5JE?Tqd38OIwq(Q_^a^g+6BT^JkYl{v9;ZK+nwYaJ4Ytg~2+50ym2OTigOS?(l#M*n1f{(XHL*mR(`1A;1kH z|NBnVmZBj#0ri2|qI}nI@6(=RWvKo5q$oLB)<8_hM@x^Q_jOolS3m1a7SC3oB6ldO z#@$rLI9_X~C;qQcrye~@($0&ny!#Lmvs>dp%W@^h=)7mVc+fL$@XTTZb5Gtx<6O?& zmfN}5P+|<&U@%7Dz){S9VuREYbON?mYhOuxLHjgeoneQS(&aK@zx-fn$-t+TTN8+Z1y zZ}2`M2yaeHzC*&S7eVJI`7ujA%6FGuM;|o0!KCOC8KDv9?-lS;flTlwkC*1T&+v;- z>XnygWoIM_%c24{X&(|Fg%ZiS8g$ z#AeRX@(VI5gmqMW7h%~;!O)yEO+D`?(g~*qq1~*X$TK?_h?TyG&^(Rdpk_|&nI}J$ zeftW_!^jNO9L~EOaYtH0@`dUS)A|5fr8C#!_ppmpRsvR!D?UDMnVdYwyy1_*QUvZy z{BfTZ_BW;?uco3Kt|U?8@TJeeHU;yRDVM^Qjwgxw2nkNzmGCfJ$clUb;g=m^&Z>jM!4(c;1FU;9&$;H!Pd^PT({Geqy{S@<9Cp|_uRWu?*fi(p9j&}%8 zk}9%XE(+yFbC?H?@d(w8Duk}P^EzG`Ru#B5vaVmcy_Cpe;>yQ)23*SYc?X4iM$^7@ zTSB4rs}seJ`=f+;Z{_77y0OKZgT34BI%b{sCY>ZuFBayT!WR!CE0&iJj&QG-8HHnD za153ul>aY`2Gv+aa%sTvxR=p>rZ5=Zx-9avxWR7oX_+*kXTZ@C$VEhC#Fd@T!)hfm z`qDk7#nx(&<-(}Iac`a2GpOWj4zGTSSNHS)m3uo3ic>4sJ1vsaO3`ghL}it7uB8=p z-X!Kg?~_5o=iw*9voQ@dHdjP*S|iqpIxDd+`+sfd4p44CFdbV;;pVWU4%-fRe_jhE zoPGMS&bnLio$A?pmt1H+AB{?X*ua0xCNlpg+JFN7gLVE&`XYgY4ULh%6fqb^$6%@P|HTam_+td{-2nuJ zMU`(ON0NWx-7nuASdrKOa^E1GOJ5(0;eyRO4c639CO%7T_bhA)*{*yu#$xgBTc4y@ zs!a?aVX#sJ{=$`n1l(Ny5HgvY>Tf6)o=y|8l1x4ARCKYNpG`2;T99}vL>ZkQz$v+t zW(t~K#nzKu&{P~aR&6el)gI2@C;{Rrz@v9OnF`a#d`1i}IwsJ~x%gsbFmGT05QE|J zR{*NkQvv`2J0SVi7=8_h(50YD529r`y1x1up^=&R<^ zn+GJv61h`c*|*dSu7-U$u3%GcB=y0uj0_nWIa{3G5o&gCvGX~H&b57-P6-bFo-{M^ zyxix^@^L;E0`$f#HD{w2s#d;GHpbidj&w;iaM=0ZEY-H3$#`2Y6_AA2NUaiz&B{@ zkzjB2oQs3so(@Ki>anyLht?#rrfq8_6tce67V}Bt%V?OH9Jq>DPOjl~_phsM0++?N zM%aW0wb`fEVW*2=OHYm#YD%AB85z&hzF0YfCwAwF6UD6M=u1ed>j!+s#Ill&)r08;!< zkbJ*yeG*U!EM@L){UfKUsVSp(NyhrBxt^!CxsyBEdT?|1khU{-0~&)>%zxxzNF)gu z-A2aR-O|n8#lzVR1VwX|7U+L>x6xw&)|8sLBM=`+>i4d{xBq{=*$s>i454c7;bwml zIEfDeL5m~c-#)*v1h}i^0aYCC-F6e+x4%RjvKeBH2wsyc903jC#Pk=C% z&W?`eAh6vH7dvYw5_D60XDgsH2Xl8j5ZK4s%^3uCb^xDMnz?A)w@ybTZ)3i#jN z3&^_zNP*q0y{w%;VC$34!A|y0)+Fd}l&oDn%^g4}z*7i>qXHsP$$`4%8Xy6nP!|N&1EFuHc{*8HySZCByIGS60!RTsS^(gJzyNe;5I|rc z*cJeyJrGd}czq27b^w8aOZEU50H7`)uqy}*012Q7pza9*dx5~$L11qZ@D1R?Jphrp zyQRIohrI(3QpyA!jS76(-pa!cpgvmc{TjOKgdu)@#PdIf`-Npr=J)^L4`A{g%x&F) z8Tu^HIQE(KA zAaE?e4{!qvU=TR)i5WT}0T|Km(e>c~U!6PwMhEKuza^5N@8LjQbVB#>Ga-R1w@?Tg zdEi4oAbuGLB`5&m2doqnu$GX(;QZ)>gn?j!ND?3cvCdE+BogQZI3xyQo&i7t06>5| z`iXuId?$*P1k2bwp2rW3w^Ab~jqU`6u)h9oEe%n6z=0Hyb9`pFYN zpJ<{0^A5;d;Q3oRnd|TG(RKbwXud$tD*E}qhyuU*5~P67lTKtIAi4n}o@hlG5EQ+C z0raCh=;UK)Ag2HVX7;2^83f!aILWDifJp{&K*&H4?1x5G0|66$@>T-`NZ^y4F2K{@ zb9x}yk1@;vx_)o_i;M7;=LTGyTAECF`@kvq|00B$GcR>1Q4o&&Vu4^{eIa01+a zQscW?0_Nr9mkqFNe&o;!?c~}gfYphPJGsFK{ICT9G8Cvy0%#mScb>HSt};QuM=K{k z?LjB^r+~`W0D< zT|memqy0~h1G}b^<|ifJ)%nQ*Brx(%#Se(8@6Eq!b3kf+|B2T4z;&UMAFja6fA7c* zSdHJy+yUOp$!C}0}?=216R*Zy7C5=%=g?4K=6Lgo!BLSfJRpU zboYYuQiU};I964=rlaO$Fs0!i5aU5yW5=U-~V z|IeC$&GWww5&vg32p`()0cxIP#KeBV42{VT$p42fn$&+3qAC6pM>#0(|7>7ha~Bsk zXYU`p|9!&$*B6@BKbw8GalXef1uNTIp$$wxOrvRi64uibut`p+^MmaCzW@T3210Y9 z&FG7Oz3bxO?BQT<0rG?h2Kz1G;F785Lu;V}P_?+Erxk$i(;cMk-355FmwV8{R z4amaW@)|JA&n|$fCj;0!JISDT^iLVu=tlp6oPLzSp!{et^jjGMZ4LfjhF$dzm>t!;^W_C2pC$3{<{nUgJ7r!gl7GBJ*XfGz4-st7AlB9OPzn0 z2?(IY^S{bq5Pm?L{JRYJD(Vj!KoCga*3R$s1Ox#_`d5DtBor<9e=9@$i7yf<_&dxH zz)<@gHwaQd;E(*T-H10=~GbOA0||1mCH0IiMx4GRMD=eP*;&f)iZK+NPn z$^b_Ay$lHu>yI)xU>W|_ACe!~ar|B;_$SPOX!-+Y6yneM1%f;O0UwJ0k97{^hyE!` zfDO_gc<}T8$s7CvXj29K_mA9#3P6Be)$e5(=12gJHYa~;i~M8F0CN0Sx_G#m+dEjh zokZ@|w)e3H^a2R1 literal 0 HcmV?d00001 diff --git a/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py b/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py new file mode 100644 index 0000000..77d2d82 --- /dev/null +++ b/analysis/pd_sep_paper_section/scripts/plot_kv_memory_wall.py @@ -0,0 +1,127 @@ +"""Decode-side KV cache memory budget as a function of per-request KV +footprint and prefill/decode split. + +Idea: PD separation is equivalent to multiplying the per-D-instance KV +demand by (N_total / N_D). For workloads with large per-request KV +footprint (agentic), this concentration breaches the memory wall. + +The plot fixes the system-wide concurrent decode count to a steady-state +estimate from the trace (QPS x avg_decode_seconds) and shows per-D-instance +KV pool occupancy as a function of per-request KV footprint, one line per +PD layout. + +All constants are documented at the top so they can be audited. +""" +import argparse +from pathlib import Path + +import matplotlib +matplotlib.use("Agg") +import matplotlib.pyplot as plt +import numpy as np + +# ---- Cluster constants (8x H20, vLLM 0.18.1) ---- +N_TOTAL_GPUS = 8 +HBM_PER_GPU_GB = 96 +MODEL_GB = 50 # Qwen3-30B-A3B MoE weights bf16 +ACTIVATION_OVERHEAD_GB = 18 +KV_POOL_PER_GPU_GB = HBM_PER_GPU_GB - MODEL_GB - ACTIVATION_OVERHEAD_GB # ~28 + +# ---- Workload steady-state ---- +# Peak QPS on the sampled trace = 1.6; mean E2E ~5s under Combined; both +# numbers from REPORT.md. So at any instant ~8 decodes are alive. +CONCURRENT_DECODE = 8 + +# ---- KV footprint constants for Qwen3-30B-A3B ---- +# 2 (K+V) * 4 kv-heads * 128 head_dim * 2 bytes * 48 layers = 96 KB / token. +KV_BYTES_PER_TOKEN = 2 * 4 * 128 * 2 * 48 # = 98304 bytes + + +def kv_mb(seqlen_tokens): + return seqlen_tokens * KV_BYTES_PER_TOKEN / 1e6 + + +# Reference operating points (per-request KV size, MB) +POINTS = [ + ("chatbot avg (~2k input)", kv_mb(2_000)), + ("agentic avg (33.6k input)", kv_mb(33_600)), + ("agentic p90 (101k input)", kv_mb(101_000)), + ("agentic p99 (132k input)", kv_mb(132_000)), +] + +# PD layouts: (label, N_D, color, linestyle) +LAYOUTS = [ + ("Combined 8C (N_D=8)", 8, "#2ca02c", "-"), + ("PD-sep 4P+4D (N_D=4)", 4, "#ff7f0e", "--"), + ("PD-sep 6P+2D (N_D=2)", 2, "#d62728", "-."), +] + + +def occupancy(kv_per_req_mb, n_d): + """Per-D-instance KV pool occupancy (fraction).""" + pool_mb = KV_POOL_PER_GPU_GB * 1024 + demand_mb = CONCURRENT_DECODE * kv_per_req_mb / n_d + return demand_mb / pool_mb + + +def plot(out_path): + fig, ax = plt.subplots(figsize=(8.5, 4.6)) + + kv_range_mb = np.logspace(0.0, 4.5, 400) # 1 MB .. ~30 GB + for label, n_d, color, ls in LAYOUTS: + y = occupancy(kv_range_mb, n_d) * 100 + ax.plot(kv_range_mb, y, color=color, lw=1.8, ls=ls, label=label) + + # memory-wall threshold (vLLM starts queuing aggressively above ~90%) + ax.axhline(90, color="#888", ls=":", lw=1) + ax.text(1.2, 93, "memory wall (~90%, vLLM stops admitting new reqs)", + fontsize=8.5, color="#666") + + # operating-point markers, labelled along the top edge + for name, kv in POINTS: + ax.axvline(kv, color="#777", lw=0.7, ls=(0, (1, 2))) + ax.text(kv, 198, name, fontsize=8, color="#333", + rotation=90, ha="right", va="top") + + ax.set_xscale("log") + ax.set_xlim(50, 3e4) + ax.set_ylim(0, 200) + ax.set_xlabel("Per-request KV footprint (MB)") + ax.set_ylabel("Per-D-instance KV pool occupancy (%)") + ax.set_title( + "Decode-side KV concentration explains the PD-sep memory wall " + f"(8x H20, KV pool ≈ {KV_POOL_PER_GPU_GB} GB/GPU, {CONCURRENT_DECODE} concurrent decodes)", + fontsize=10, + ) + ax.grid(True, alpha=0.25, which="both") + ax.legend(loc="upper left", fontsize=9, framealpha=0.95) + fig.tight_layout() + fig.savefig(out_path, bbox_inches="tight") + plt.close(fig) + print(f"wrote {out_path}") + print(f" KV/token: {KV_BYTES_PER_TOKEN/1024:.1f} KB " + f"pool/GPU: {KV_POOL_PER_GPU_GB} GB " + f"concurrent decodes: {CONCURRENT_DECODE}") + print() + print(f" per-D occupancy at each operating point:") + print(f" {'workload':28s} {'KV/req':>10s} " + f"{'Combined':>10s} {'4P+4D':>10s} {'6P+2D':>10s}") + for name, kv in POINTS: + c8 = occupancy(kv, 8) * 100 + c4 = occupancy(kv, 4) * 100 + c2 = occupancy(kv, 2) * 100 + print(f" {name:28s} {kv:>7.0f} MB " + f"{c8:>9.1f}% {c4:>9.1f}% {c2:>9.1f}%") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--outdir", default="analysis/pd_sep_paper_section/figures") + args = ap.parse_args() + out = Path(args.outdir) + out.mkdir(parents=True, exist_ok=True) + plot(out / "fig_kv_memory_wall.pdf") + + +if __name__ == "__main__": + main() diff --git a/analysis/pd_sep_paper_section/scripts/plot_workload.py b/analysis/pd_sep_paper_section/scripts/plot_workload.py index e5fc37e..f1be864 100644 --- a/analysis/pd_sep_paper_section/scripts/plot_workload.py +++ b/analysis/pd_sep_paper_section/scripts/plot_workload.py @@ -160,29 +160,39 @@ def plot_reuse(rows, out_path): d = reuse_decomposition(rows) total = sum(d.values()) parts = [ - ("intra-session reuse", d["intra_session_reuse_tokens"], "#2ca02c"), - ("cross-session reuse", d["cross_session_reuse_tokens"], "#1f77b4"), + ("intra-session reuse", d["intra_session_reuse_tokens"], "#2ca02c"), + ("cross-session reuse", d["cross_session_reuse_tokens"], "#1f77b4"), ("first emission (reused later)", d["first_emission_will_reuse_tokens"], "#ff7f0e"), ("unique (never reused)", d["unique_no_reuse_tokens"], "#d62728"), ] - fig, ax = plt.subplots(figsize=(8.5, 1.9)) + fig, ax = plt.subplots(figsize=(9.0, 2.2)) left = 0 + handles = [] for label, val, color in parts: frac = val / total - ax.barh(0, frac, left=left, color=color, edgecolor="white", height=0.6, label=label) - if frac > 0.025: - ax.text(left + frac / 2, 0, - f"{label}\n{frac*100:.1f}%", - ha="center", va="center", fontsize=8.5, color="white") + b = ax.barh(0, frac, left=left, color=color, edgecolor="white", + height=0.55, label=f"{label} ({frac*100:.1f}%)") + handles.append(b) + if frac > 0.04: + ax.text(left + frac / 2, 0, f"{frac*100:.1f}%", + ha="center", va="center", fontsize=10, + color="white", fontweight="bold") left += frac ax.set_xlim(0, 1) + ax.set_ylim(-0.6, 0.6) ax.set_yticks([]) - ax.set_xlabel("share of total cacheable tokens (block-aligned, 512 tok blocks)") - ax.set_title("Where do prefix cache hits come from? " - f"(N={len(rows)} requests, sampled trace)") - ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.45), ncol=4, fontsize=8, frameon=False) + ax.set_xticks([0, 0.25, 0.5, 0.75, 1.0]) + ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"]) + ax.set_title( + f"Where do prefix cache hits come from? (N={len(rows)} requests; " + "block-aligned 512-tok blocks)", + pad=8, + ) + ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.20), + ncol=2, fontsize=9, frameon=False, handlelength=1.5, + columnspacing=2.5) for spine in ("top", "right", "left"): ax.spines[spine].set_visible(False) fig.tight_layout() diff --git a/analysis/pd_sep_paper_section/system_analysis.md b/analysis/pd_sep_paper_section/system_analysis.md new file mode 100644 index 0000000..1a03059 --- /dev/null +++ b/analysis/pd_sep_paper_section/system_analysis.md @@ -0,0 +1,226 @@ +# Why does PD separation fail on agentic workloads even though prefill stays compute-bound? + +This is the central paradox of the section. The roofline in +`figures/fig_c6_roofline.pdf` shows prefill is compute-bound at every +realistic reuse level — at 95 % cache reuse, arithmetic intensity is still +≈ 4,500 FLOP/byte, more than two orders of magnitude above the H20 ridge of +37. The DistServe / Splitwise argument follows directly from that fact: +"prefill is compute-heavy, decode is memory-bound, so isolate them onto +different GPUs and specialize each." + +Yet on this workload, single-machine PD separation regresses TTFT by 72 % +(REPORT.md §3.1) and saturates the decode-side KV pool at 97 % occupancy. +This document explains, layer by layer, why a true premise (compute-bound) +does not imply the conclusion (PD separation pays). All five layers are +backed by either figures in this directory or measurements in +`analysis/pd_separation_analysis.md`. + +The short answer: **the roofline tells you about the per-kernel efficiency +of prefill. PD separation is a decision about a whole serving system. The +gap between those two scales is where DistServe's argument loses force — +and where agentic workloads, with their very large per-request KV +footprint, push the system past a memory-capacity wall that chatbot +workloads never reach.** + +--- + +## Layer 1: compute-bound ≠ "needs dedicated GPUs" + +Roofline analysis classifies a *kernel*. It answers the question, "given +that this kernel is running, is it bottlenecked by FLOP rate or by HBM +bandwidth?" — it does **not** answer: + +- how long the kernel takes in wall-clock terms, +- whether two kernels can profitably share a GPU, +- whether moving the kernel to a different GPU makes it faster. + +PD separation needs the second and third answers, not the first. A 50 ms +compute-bound prefill burst can perfectly well coexist with decode steps +on the same GPU; you lose at most a fraction of a decode step's latency +per chunk. Co-location only fails when prefill bursts grow long enough +that decode requests starve. + +The DistServe paper's roofline argument is a *necessary* condition +("prefill *can* be compute-bound, so dedicating GPUs is *not wasted*"). It +is **not** a *sufficient* condition ("therefore dedicated GPUs pay"). + +## Layer 2: in agentic, absolute prefill work after cache hit is small + +The roofline is computed in `figures/fig_c6_roofline.pdf` at a full 64 k +context. But the operating point on the trace is shifted by prefix cache +hits: + +| reuse | new tokens | prefill time @ ~7,000 tok/s | +|---|---|---| +| 0 % (turn 1 cold) | 64,000 | ~9 s | +| 71 % (trace average) | 18,600 | ~2.6 s | +| 95 % (deep multi-turn) | 3,200 | ~0.5 s | + +Average-case prefill is ~2.6 s of compute. With 8 GPUs and peak QPS 1.6, +each GPU sees ~0.3 s of prefill work per second of wall-clock. Chunked +prefill in vLLM slices this into ~8 k-token chunks of ~50–100 ms each, then +yields to decode. The decode-side disturbance per HEAVY request is on the +order of "a few hundred ms of stretched decode," not "seconds of stalled +decode." + +PD separation, in its best case, eliminates this disturbance. The +*ceiling* on the benefit is therefore: hundreds of ms per HEAVY request. +This budget has to absorb everything PD separation costs. + +## Layer 3: PD separation relocates compute; it does not accelerate it + +A prefill kernel does the same FLOPs no matter which GPU runs it. PD +separation's potential acceleration vectors are only two: + +1. **Larger prefill batch** → better SM utilization for the prefill MMA + kernels. +2. **No chunked-prefill yield to decode** → no overhead per chunk handoff. + +Both are quantitatively negligible in this regime: + +1. At peak QPS 1.6 with ~2.6 s of prefill per request, *system-wide* + prefill concurrency averages to ~4 active prefills. A 4P split sees ~1 + prefill per GPU at any moment, so batching gains are zero. A 6P split + makes it worse, not better. The roofline ceiling of 148 TFLOPS is + already reachable at batch=1 for sequences this long. + +2. Chunked prefill's per-chunk overhead is dominated by scheduler tick + time (≈ 1–2 ms), not the chunk transition. Removing it saves single + percentages of prefill time. + +So the *speedup side* of PD separation is bounded by the few-hundred-ms +budget from Layer 2 and contains no hidden upside. + +## Layer 4: the costs of PD separation are workload-scaled + +PD separation adds two costs, both of which scale *up* with workload size: + +1. **KV transfer over the network.** Mooncake transfers KV block-by-block + after the full prefill completes (no layer-wise pipeline; see + `analysis/elastic_hypotheses.md` H5). Empirically, transfer takes + ~1.1 s p50 for HEAVY requests (~40 k tokens of KV ≈ 3.8 GB at + 96 KB/token), with tail extending to 18–30 s. Transfer time grows with + context length. + +2. **Decode-side KV concentration.** All decode work is funneled onto a + subset of GPUs (4 of 8 in 4P+4D, 2 of 8 in 6P+2D). Per-D-instance KV + *demand* therefore scales by N_total / N_D. This is the killer cost; + Layer 5 quantifies it. + +Both costs scale linearly or worse with per-request KV footprint. KV +footprint, in turn, scales linearly with input length. So PD separation +gets *worse* exactly along the axis (long context) where the workload is +moving. + +## Layer 5: the decode-side KV memory wall (the actual mechanism) + +Visualized in `figures/fig_kv_memory_wall.pdf`. The model is simple and +its constants are auditable in +`scripts/plot_kv_memory_wall.py`: + +``` +per-D occupancy = (concurrent_decode × KV_per_req) / (N_D × KV_pool_per_GPU) +``` + +with: + +- `KV_per_req` = `seqlen × 96 KB/token` for Qwen3-30B-A3B + (2 × 4 kv-heads × 128 head-dim × 2 bytes × 48 layers = 96 KB/tok) +- `KV_pool_per_GPU` ≈ 28 GB (96 GB H20 HBM minus weights and activations) +- `concurrent_decode` ≈ 8 at steady state (peak QPS 1.6 × mean decode + duration ~5 s under Combined) + +Plug in the trace's input distribution from +`figures/fig_c1a_io_cdf.pdf`: + +| operating point | KV/req | Combined (N_D=8) | 4P+4D (N_D=4) | 6P+2D (N_D=2) | +|---|---|---|---|---| +| chatbot avg (2 k) | 197 MB | 0.7 % | 1.4 % | 2.7 % | +| **agentic avg (33.6 k)** | **3.3 GB** | **12 %** | **23 %** | **46 %** | +| **agentic p90 (101 k)** | **9.9 GB** | **35 %** | **69 %** | **138 %** ⚠ | +| **agentic p99 (132 k)** | **13.0 GB** | **45 %** | **90 %** ⚠ | **181 %** ⚠ | + +vLLM's scheduler stops admitting new requests at ~90 % KV pool occupancy, +so anything above the wall translates directly to queueing. + +Two consequences fall out of this table: + +1. **PD-sep with even a 4P+4D split breaches the wall at p99 context.** + p99 alone is ~1 % of requests but holds the GPU for tens of seconds of + decode, so its KV stays resident; over a long enough window the wall + gets hit even from the tail. With 6P+2D the wall is breached well + before p90. + +2. **For chatbot, the entire table sits under 3 %.** PD separation never + approaches the wall because chatbot per-request KV is 15× smaller. This + is the assumption DistServe inherited from its target workload, and + the assumption that silently breaks under agentic. + +The empirical KV occupancy on the 6P+2D run was 97 % +(`analysis/pd_separation_analysis.md` §3.3) — the model and the +measurement agree to within the resolution of the steady-state assumption. + +## Layer 6: the DistServe / Splitwise assumption that silently breaks + +To formalize: the regime in which PD separation pays is bounded by both a +roofline condition *and* a memory-capacity condition: + +| Condition | Form | Chatbot | Agentic | +|---|---|---|---| +| Prefill is compute-bound | AI_prefill ≫ ridge | ✓ | ✓ | +| Decode is memory-bound | AI_decode ≪ ridge | ✓ | ✓ | +| Per-D-instance KV demand fits | concurrent × KV/req / (N_D × pool) < 1 | ✓ (≪ 1) | ✗ (>1 at p90+) | +| KV transfer time ≪ saved interference | transfer_s ≪ saved_decode_stall_s | ✓ (KV is MB) | ✗ (KV is GB) | + +DistServe and Splitwise hold all four conditions implicitly in their +short-context regime. Agentic violates the bottom two. Both violations +have the same root cause: per-request KV footprint is 15–60× larger. + +This is the falsifiable claim of the section: **PD separation pays iff +per-request KV footprint × decode concurrency stays well below the +per-D-instance HBM pool. When that condition fails — and it fails +unavoidably for long-context agentic workloads — PD separation is net +negative regardless of how compute-bound prefill is.** + +The roofline doesn't tell you whether you're inside this regime; only the +memory budget does. + +--- + +## What this means for the paper section + +The figures we already have support this argument: + +- `fig_c1a_io_cdf.pdf` — establishes the input-length distribution + responsible for the large KV footprint (p50 33.5 k, p90 101 k, p99 + 132 k). +- `fig_c1b_reuse.pdf` — establishes that 79 % of reuse is intra-session, + i.e. the request set has long-lived sessions whose KV must sit in the + pool through many decode steps. +- `fig_c6_roofline.pdf` — establishes the prefill compute-bound fact. + This is the apparent contradiction the section resolves. +- `fig_kv_memory_wall.pdf` — establishes the resolution: the memory budget + is what governs PD separation's viability, not the roofline. +- `fig_c7_routing_lever.pdf` — establishes that cache-aware routing + recovers most of the benefit PD separation promises, without paying the + memory-wall cost. + +Missing for a rigorous re-grounding (deferred until the cudagraph re-run +matrix lands): + +- Per-step decode KV utilization time-series from a live PD-sep run + (currently inferred from a single log snapshot of "Running: 0, Waiting: + 6, KV cache: 97.1 %"). This would *directly* show the memory wall being + hit instead of relying on the steady-state model. +- Per-request TTFT stacked breakdown (prefill, KV transfer, decode-side + wait) on the new trace; currently `analysis/pd_separation_analysis.md` + §3.3 has it on the old methodology. +- CUDA-graph ablation: with `--enforce-eager` removed, PD-sep's D-node + could in principle close some of the per-step decode latency gap. The + Layer 5 model is gate-independent — wall demand grows with concurrency, + not per-step latency — so this should not change the conclusion. But + the section needs the measurement to say so honestly. + +The 4 h cudagraph experiment matrix (Combined / PD-sep × eager / +cudagraph × 3 seeds) on `traces/w600_r0.0015_st30.jsonl` would settle +those three items.