Archive for the 'ruminations' Category

ASRock C224/C226 + Brocade 1020 = … this is why

‍‍כ״ד ניסן ה׳ תשע״ד - Wednesday, April 23rd, 2014

This is why this build out process has taken so long (and why I have to hold on to so many damn boxes). Only this time, it’s not just my frustrations at side problems that arise and derail; this one could hurt someone.

On Saturday night, I planned to use a combined server/workstation build intended for Miami to perform a mirror copy of my main 5TB image store, and then go out. I didn’t leave the house for another 24 hours.

The machine has an ASRock E3C226D4i-14s board currently running a E3-1245v3, and to accommodate the ITX “plus” motherboard, and to allow me to over deploy storage to a site 1,500 miles away, I chose a Silverstone DS380 case which has eight hotswap 3.5″ bays. Seeing as I have to make/seed a few of these copies, and that integrity is critical, I chose to use this build to mirror off my DAM intake station. Initially, I left it as is, and tried to get a bit of a throughput boost by teaming the onboard NICs. I was getting 1.1Gbps, but the source array can perform much higher reads and even the write array can sustain 130+MB/s writes.

To speed up the process, I decided to stop and replace the Quadro K600 with a Brocade 1020 that had been sitting around for a few weeks (and worked just fine before that) and move the machine next to the DAM station, and run it by direct SFP+ connection, with the added benefit of putting on a UPS backed outlet. Even if I chose to do two copies at once, at least I’d have that benefit.

In short, Windows booted twice with the card – and much slower than expected, the first time I grabbed the full Brocade management pack off the DAM server, installed and rebooted. On the second boot I started a simple SMB file copy of ~5TB and within seconds, bluescreen. Pulled that card, replaced it with a working one from an ASRock C2750D4I. Slightly different revision. Same deal.

What followed was hours of reboots and simple checks and permutations that are both inane and necessary.  By 10AM – Sunday now – yet to sleep, I started a very long and organized email to ASRock because I either had a defective board or there was a firmware problem. I am convinced of the second at this point.

I had certainly noticed that my temps were at least 12C higher – sometimes 20C – on the Brocade (8.5W) than a sealed case running with the Quadro (41W), and certainly the 300W+ draw on a UPS that sits at 230W with NEC PA241W, the DAM station, and whichever Haswell+Quadro machine is attached, all active. I attributed this in part to the Brocade stressing the system in some sort of processing lock and decreased airflow with the case open. At its worst, I believe I saw mid-60s, but by then I was done actively diagnosing and instead organizing my write up and grabbing screenshots.

Monday arrived; so did a CPU for an E3C226D2I compact workstation build I wanted to try. The intended case is a Norco ITX-S4, essentially a half height, half RAM version of the E3C224D4I-14S build with no server responsibilities or permanent place; I’ve even considered 12V and some inbuilt battery options.

Unfortunately, I ordered a 1265v2, not v3. So I pulled the Xeon E3-1230v3 (and RAM) from my test bench Supermicro X10SL7 and went about putting a lot of cables and drives in a very small case.

The first concern, of course, was heat. With only a single 80mm providing exhaust and a Noctua NHL9i HSF I was concerned about airflow. I decided to boot from IPMI and watch the temps. To kill two birds with one stone, I decided to go directly back to now very (still) overdue task of making a mirror of my image archive/composite work, so I went to use the 2nd Brocade 1020. I plugged it in, hit the power button and walked over to my workbench station.

In the dashboard, I noticed two things:

  • crap: preview window was sitting at the Brocade OpROM splash
  • double crap: my temps with less than a minute uptime were ~67C.

So, at this point, it’s 3AM Tuesday morning so I go to the local bar for a drink and to vaguely rethink shuffling parts around, because this build wasn’t going to work.
Not exactly.

What I didn’t realize that while my temps are of course higher – they aren’t that high.  Right now the C226 has been steady at 45C/42C for hours with 2x8GB DDR3L ECC, the Quadro K600, 4x4TB drives, and Seagate 600 Pro 240MB.

Norco C226+K600

This build is completely feasible for its intended use, though I may drill holes to replace the 80mm (already swapped for a Noctua 80mm PWM) with a 92mm and bolt on a 120mm outside the case behind it.

So, why didn’t I immediately jump and say “oh, it’s the Brocade being mishandled again” when I’m having obvious boot problems?

Because the problem is slightly different.  The E3C224D4I-14S surges and locks up the system completely: for example, keyboard LED lights, whether by PS/2 or USB, will not toggle.  And if I see a power surge, after no more than thirty seconds, but usually just a few, power draw drops to (slightly below) normal levels because most of the system is halted.  As a result, temps took a lot longer to climb on the C224; sure, I was seeing 47C/53C  CPU/MB to start, but it took hours of rebooting, sometimes with the card pulled, to get to 60+C temps.  Also “luckily” the E3C226D2I handled its problems “better” – the system wasn’t locked, I was free to hit ‘x’ and watch the OpROM load all over again.

And keep cooking.  In the minute it took me watch the boot process just from LEDs to leave my desk area and head over to the workbench with the IPMI page, it got 22C hotter than it’s running right now – after hours of uptime with a “hotter” card.

So now it’s Tuesday – yesterday – and I’m doing similar fiddling.  I pull the card and install Windows just fine.  Since I never had a hard lock with the Brocade, I decide to give it another shot.  Incidentally, it was at this point that I saw that ASRock got back to me and I’m looking at my phone.
It was where I also got very lucky.
My air conditioner turned on.
My lights turned off.

And of course, now everything in the room is beeping: the desk APC, the rack Eaton 5125, two sets of desktop speakers and three tablets and two phones with emails from Spiceworks, SNMP, the Eaton UPS directly, from whatever other gremlins that may roam – fun.  And on the way back from resetting my circuit breaker I walk by my workbench station, open to the ASRock’s IPMI page.

77/79 CPU/MB

Yeah, I ran to pull the plug.  My UPS was happy to supply power as my little 8 inch computer with $900 in HDDs alone roasted itself with no end in sight, straight through the power event.  Had I just sat there – hey, the system’s still responsive, don’t fiddle, give it a minute and observe… or worse – avoid the temptation to fiddle – let it sit for a bit and go to the store… this is a Manhattan apartment building.  I have old 50s wood floors.  There are real worst case scenarios that transcend “off site backup.”

What’s also scary is that this card is not new, well established, tested in turnkey and big contract custom deployments, used in conjunction with boards from IBM, Dell, rebranded, etc – it should represent an in spec PCIe as a card can be, so it raises the question, what else could trigger this?  Even if the answer is “nothing,” consider that is recently in vogue at some of the advanced home networking forums because of eBay liquidations plus the curiosity surrounding some really interesting and compelling motherboards that ASRock is selling – it’s not a huge number of people, but still: people.

So that’s why this, that’s why the emails, the blog post, the hours upon hours of walking this through and overly detailed emails, and presenting everything as best as I have recorded or done, errors of distraction, sleep deprivation, and all.

Oh, and to completely rule out the card: that second Brocade card, the working pull, went back into the ASRock Avoton, booted fine, installed Windows fine, and managed 2+Gbps for a few seconds (locking up the whole UI) but recovered without error to do a 100MB/s file copy over 65GB+, and then to reduce IO bottlenecks, LANBench where it did 2.7+Gbps.  Less than an hour after being cooked at 80C.  It’s not the card.
Both ASRock builds shared nothing other than the 4x4TB drives (and only briefly, to reduce risk and diagnostic complexity)… and ASRock Haswell Xeon firmware.

The boards are otherwise great, doing exactly what I want, giving me options that no one else really makes.  ASRock knows how to make motherboards.  That’s not the question.
But something is very wrong here.

While I’m working on my comeback post(s)

‍‍ד׳ ניסן ה׳ תשע״ד - Thursday, April 3rd, 2014

Here’s what the last 7 months looks like, just from the hardware side.  No idea why I obsessively did this now (well I do, outgrowth of one long explanatory Facebook PM and one phonecall about a potential future project/endeavour) because I have had three places to be.  Hardware for site 2 is only partly deployed (but assembled and 90% configured) but will be done next week and site 3 hardware (other than the AP/routers which were done three years ago) is built but is being used for testing and will deployed sometime in the next few months.  Travel is difficult with Athena.

Making this rundown a page rather than a post as it’s in flux and it, like life, are always subject to and the subject of change.
Traditional photo equipment (think enlargers and development, not cameras and lenses) and fabrication/power tools which come after this is done have seen some work and a month or two of time, but are excluded for now.

A Nikon system for all bodies, all focusing methods, all lighting situations (or: how to get me to post here – write long answers to Facebook questions.)

‍‍כ״ג סיון ה׳ תשע״ג - Friday, May 31st, 2013

In response to this question in the F2 group on Facebook.

Did a quick run up through that list, and his price would be consistent with dealers selling those items used (without adding in smaller items, prices on the MD2+MB-1 and PB/PS-6 can vary a lot, didn’t bother with the Vivitar; it’s not really worth much) at roughly Adorama’s Ex- or bettter cosmetic KEH BGN items my running tally on my calculator was 3050. Of course, you would get a 3 or 6 month warranty there, respectively, plus grace return, but you would not likely get boxes.

It’s neither here nor there – much will depend on what you want. I own much of these, have tried, rejected some. Here are some notes, FWIW, Bjorn Rorslett is a good subjective resource; I only disagree with one or two of his assessments. (I ended up reordering these by focal length to consider overlap.)

20 2.8 AIS I’ve handled it, some in store images, played with a freind’s copy; never felt the desire for it. While Rockwell and Rorslett end up giving it a similar “rating,” I think Rockwell’s description of it as “typical wide Nikkor prime” (barrel distortion, veiling flare wide open, etc.) is dead on; better “technical” performance will be found in later designs.

24/2.0 AIS This feels a lot like the 35/1.4 to me, only it take even longer to sharpen up and dispose of flare; it is a lens I’ve tried and don’t own.

28/2.8 AI It’s a dated, albeit fine lens, but the AIS close focus is better (for my purposes) and other than price ($150), I would skip both of these and go for the real gem of the MF Nikkor wides – the 28/2.0 AIS.

35/1.4 AI (mine is AIS; identical as far as I know, other than the standard mechanical change of focus throw/grease and 7 vs 9 (AIS) blades.) I use this lens out of necessity; had the Samyang or Zeiss been out at the time, I would own those. Very bad CA on FF digital. AIS copies I’ve seen tend to have Schneideritis, haven’t seen enough AI copies to speak to that.

50/1.4 AF(D?) Nothing specific to say about this; it’s very much a typical post AI 50/1.4. Manual focus will be a weakness. With respect to other options, it doesn’t have the technical distorion flatness of the 50/2.0 AI, the low light abilities of a 58/1.2, the romance and sheer T/stop of the 55/1.2, the modern utility of the CV 58/1.4 SLII, the unique formula of the 5.8cm/1.4, the pastels of the 50/1.4 non-AI SC, the modern abilities of the 50/1.4G or (and fantastic non-vignetting of the) Sigma 50/1.4 – of course, these would be useless on all but the last AF film bodies. Yes, I have a lot of normal lenses for 35mm.

85/2.0 AI (I think I have the AIS) Bjorn is correct on his claims of a “grey” rendition; Ken Rockwell is correct about the handling of this lens; it’s almost exactly the same barrel, size, and weight of the K style non-AI 50/1.4. If you need a compact 85 to go with a FG or FM bodies, this is your one choice. But it has a rendering style that doesn’t do much for me.

85mm/1.8 AF If this is the non AF-D version, then this is the first AF Nikkor I ever owned, bought from B&H used, I think just after they moved into the “new” store on 9th Ave. It is really a nice lens, I think formally better than any of the other slow 85mm options and a 9 blade AF lens at that. However, this really is ever slightly larger than the worst AF focus rings Nikon has made (70-210/4, I’m looking at you) and it is pretty bulky given the maximum aperture; this also makes the screwdriver AF particularly slow.

80-200mm/2.8 AF-ED If this is the one ring/push pull version, I hate it. Whether 1 or 2 ring zoom designs are better for manual focus, that one ring design is not. Now the 2 ring version, well, consider that in 1996 that was the top of the line tele zoom and remains for sale today, unchanged, outlasting numerous successor models. Even by the standards of manual lenses, this is a decent manual focus feel. I haven’t used this on my D800E, so I don’t know how well it holds up there, but if you want to avoid G lenses, what else are you going to use – often for $500 used?
***NOTE*** The M / A ring on the 2-ring tends to slightly fracture and can completely break. Be gentle with it; you may need to encircle and support the ring with your thumb and index finger to properly rotate it.

Wow, I just meant to write the 1st paragraph. If you knew all this, well, I hope it’s useful to someone. I don’t know if you are using this to jump start a working Nikon collection on an F2 body, (this doesn’t feel like a shelf collector’s set) whether you plan to use this on MF and AF film bodies, digital Nikons and/or Canons (I do all those). Now, this is obviously in light of my use (tending towards available light, MF, and avoiding G lenses), but I think there is too much overlap here; and some more modern, cheaper options. For that person, well, I would spend my money today on some combination of these:

Nikkor 17-35/2.8 AFS – $800 and forget about the below
Zeiss 24/2.8 – 500(ZF)-800(ZF2) though not a huge fan, no CRC but close focus
*Nikkor 28/2.0 AIS -$500 CRC and really good at it.
Samyang 35/1.4 – $450 certainly over my 35/1.4 AIS
Nikkor 35/2.0 AIS – $250 budget option, no CRC

40-58mm – Depends on purpose.  Always keep one of the AI/AIS 1.8 low distortion ones around for “in the field” copy type work $100.

*Samyang 85/1.4 – $260 (Yes, really.  One lives on one of my F2AS bodies.)
*Nikkor 105/2.0 DC – $700-$1000 (watch for decentering – may be wise to buy new, best manual focus on an AF lens, very misunderstood)
Nikkor 80-200/2.8D – As needed, above.

Some other older notes from a while back (though considered for a 5DII, many Nikkors though).

outgrown (facebook) status

‍‍ב׳ אייר ה׳ תשע״ג - Thursday, April 11th, 2013

finding sorting old clothing fascinating- as uncertain numbers and intended fate dictate placement, variations on folding affect the interaction of placements, object valuation based – and weighted – between functional utility, social utility, and personal totemism — of memory, of self indulgence, interplay between a systemic approach and the feel, the music playing on Pandora: its distraction and relation to memory, and the subsumption of all – and me – to the end task.  Should have done this sooner.

far more interesting than most social interaction in recent memory and the repetitive, inevitable discussions the apologia, leading EITHER to confrontation borne of ego and the bludgeon of validation or worse, bullet point, wielded OR the prophetic cloak of inspiration being thrust upon me.  and still it infects me as i run through those thoughts in their broad curves and the precise turns of recent iterations.  and letter to score unneeded conflict where I’m in the right compose themselves once more for the the thousandth fragmented time. but still, this, this, these clothes, unlike people, I can’t so easily predict and I’ve seen each of these threads before, but until this job, never the tapestry.  also, my Zepp channel on Pandora is pretty good.  To wit – How Many More Times is on and I’m out.

Before a bath

‍‍כ״ט שבט ה׳ תשע״ג - Friday, February 8th, 2013

What then is the value of a dialectic of intertextual reference?  Pretension, maybe, certainly true at times, but usually a charge from the illiterate egoist.  Validation, (either or both) establishing the credentials (of entry, consideration, engagement) of the author and a guarded gate in the name of esotericism – only the chapter headings, and not before anyone save the rare meritorious ear.  Perhaps it is the popular version of the latter; a method of revered scholars as emulated by followers, granted prestige as dialect, but stripped of its mysterious functionality, or at least lowered to code switching of a sort – putatively “Number 34″; “You didn’t tell it right” or conversely(!), a popular register – “at Tanagra” – that the cautious scholar imbued with new functionality to suit his exclusionary purpose.  In all of these, it could merely be the value of time vs. space, the efficacy of a cultural lookup table, or (perhaps as exactly here) where there is no common understanding as drawing upon diversity in basic references from often mutually exclusive groupings.  If then, if me(?), then a smattering of all, and Google and critical notes and if kind and of a popular mindset, href, remedies much.
(more…)

half time game plan

‍‍ד׳ טבת ה׳ תשע״ג - Monday, December 17th, 2012

While this entire site needs a revamp,  one thing, perhaps the most important thing, needs to be addressed – content.  While it’s obvious that I rarely post here, I do post long comments, some article length elsewhere.  I’m going to be moving copies here and revising them (not all sites allow editing and I have this silly habit of using my Nexus 7 tablet for long posts – not conducive to avoiding typos both foreign and autocorrect).

The first selection is my comment on Vice’s Art Basel article – perfect because I came to Miami specifically for Art Week, had a nearly perfect successful couple of days (I showed up late and last minute), was appreciated to the degree that talented artists on at least three continents sent me Facebook friend requests, has raised (some confused) commentary elsewhere on the web, and poetically, today is my last full day here, in my occaisional other city, and sets the stage for a more public presence (and work, always work) when I return to NYC on Tuesday.