Category Archives: hacking

Arch Linux Update Script

Hacking Fan Speed on Dell PowerEdge Servers

A couple of weeks ago, I acquired three old Dell servers to play with: a PowerEdge 840, PowerEdge 830, and a PowerEdge SC 430. One thing I didn’t think about before I got them was how much noise they might make. I’m used to desktop machines; they’re designed to run quietly enough not to annoy someone trying to work in the same room. Dell doesn’t go to the same trouble when they design their servers. The SC 430 is reasonably quiet (Dell based it on their Precision platform), but the other two, the 830 and 840, are just loud enough to be annoying.

My first thought was that I might be able to control the fan speed, but fan speed is firmly under the control of the BMC (Baseboard Management Controller); I can’t control it from software. I determined after some Googling that most people solve the problem by replacing the system fans with slower, quieter, models. Unfortunately, I also found that the slower fans often trigger the “Lower Critical Threshold” (they spin too slow), causing the BMC to spin them up, which gets me back to the noise problem I had in the first place.

Of course, because this is the Internet and I’m not the only hacker who likes to play with hardware, someone else had already solved this problem.

TLDR; the thresholds in used by the BMC can only be changed by hacking the BMC firmware update package. I downloaded that guy’s Python script and ran it on my PE 830. The script was able to parse the BMC firmware update for the PE 830, so I went ahead and ordered a new fan.

Now, before I could replace the fan, I had to deal with Dell’s custom pinout (because their 4-pin arrangement is clearly superior to the standard 4-pin arrangement that carries the exact same signals). This is already documented in several places on the web, but just to get it up here one more time:

Signal	Dell Color	Standard Color
PWM	Blue	Blue
RPM	Yellow	Green
+12V	Red	Yellow
Ground	Black	Black

I lopped the connector off of the original fan (at least it’s not a non-standard pinout on a standard connector). I couldn’t shove the stranded wire into the new fan’s connector, so I soldered some solid wire from a bit of spare CAT 6 cable onto the leads. I pushed the wire all the way through the connector and bent it over, which should keep everything in place.

Before clamping and soldering.

I plugged it in and started the machine. Good news: it was nearly silent. Bad news: loading the machine runs the core temp up to 60C (Intel says to keep it below 63). The fan I bought pushes 74 CFM at full speed; the OEM fan managed 150 CFM. That wouldn’t be a problem, except that Dell was cheap on all of their tower chassis in the mid-2000s and made the back case fan do double duty as the CPU fan.

I ran an experiment to try and determine how fast my fan needed to run to be effective. I wrote a script to collect fan speed and CPU temperature every 2 seconds. After collecting 5-10 minutes of data at idle, I started a program that fully loaded the CPU for several minutes, and continued collecting data until the system returned to a stable idle state. I ran this experiment on the PE 830 (Pentium D 940, 3.2 GHz, 130W TDP) and the PE 840 (Core 2 Duo E6400, 2.13 GHz, 65W TDP).

As it turns out, the BMC isn’t intelligent enough to vary the speed of the fan based on CPU temperature. On the new fan, it drops the fan speed in increments of 75 RPM until it gets below the threshold, then spins it back up to a much higher speed and repeats the process. This cycle is apparently unaffected by CPU temperature:

(Note: The label on the X-axis should read “Time (mm:ss)”, but I’m too tired to go back and change it now. Click on the plots for full-sized versions.)

The CPU temp peaks around 60 C. The heating seems to be fastest when the fan runs below 1000 RPM.

Now, even at high speeds, the new fan is very quiet, so noise is no longer a problem. However, the fan doesn’t cool the CPU effectively at speeds below about 1000 RPM, causing the CPU to heat very quickly if it’s loaded at the lower part of the fan’s cycle. This problem was easy enough to solve though. Instead of hacking BMC firmware hack to lower the speed threshold, I hacked the firmware to raise the threshold to 1000 RPM (the Python script already allowed this).

With the threshold set at just below 1000 RPM, the fan speed kicks back up before the CPU temp can rise too far.

For reference, here’s a plot of the fan behavior with the stock fan and stock firmware. The fan speed really doesn’t vary at all, regardless of CPU temperature.

I also tested out the new fan in the PE 840, and gathered similar results. CPU temperature still doesn’t factor into fan speed.

Fan speed is flat. CPU temp isn’t.

I haven’t bought a second fan for the PE 840 yet, and I’m not sure if I will. For some reason, it doesn’t seem as loud as the 830, even though both run the fan at the same speed.

I really wish I knew why the firmware keeps trying to lower the speed on the new fan. My best guess is that Dell’s PWM fans don’t work quite the same way as standard PWM fans (because re-inventing PWM obviously makes sense…).

M2 update

Leave a reply

This post is a follow-up on my previous post, We have a MakerGear M2. We’ve spent a fair amount of time tuning the print settings and experimenting with different printing and design techniques.

We started printing gear cubes pretty early on (http://www.thingiverse.com/thing:10483, not sure if that’s the exact model we’ve been using); we’ve printed about 3 at this point. As far as difficulty, they aren’t very intricate, but the overhang occasionally causes problems. These are probably the coolest things we’ve printed; they’re a lot of fun to pick up and fiddle with.

This particular version prints the gears with the pins already attached. We’ve found that the versions that have separate pins tend to work better, if only because the pins are easier to replace when you break them trying to assemble the thing.

Of course, we have run into a number of problems, most of them small. The above picture shows two things. First, these are the separate pins I mentioned. Second, the head occasionally catches the model it’s printing and we end up with a pile of goop (this happens very rarely). Now that I think about, third, we’re printing on painter’s tape. The models adhere to the tape better than the glass or to kapton tape. We usually only use the blue tape for smaller models though, as models with a large surface area end up bonding really, really well.

Probably the most useful thing that we’ve printed is a doorstop. We like to keep the lab open, but the door has an automatic closer on it, so we need a doorstop. Getting a doorstop wasn’t really a problem, but the carpet in our lab is slick enough that the door pretty often would push the doorstop across the carpet until it was half closed. We designed and printed a doorstop with spikes and solved the problem (http://www.thingiverse.com/thing:64022).

If you’ve been following Aggregate.org, then you’re likely familiar with MOG and our maze. We’ve printed lots of these mazes at various scales. We printed some small ones, inserted ball bearings, and sealed some plastic over the top. They made pretty good handouts for the College of Engineering’s E-Day.

(Credit to Paul Eberhart on these photos)

The above is a picture of the bottom of one of the mazes. We’ve developed a cool debossing technique that allows us to print images and letters on the bottom faces of objects.

A large part of the reason (excuse?) for buying this printer is that we can use it to print camera parts. To do that, our advisor needs to be able to print very thin layers with great detail. As a test, he started printing owls (again, not sure whether that’s the exact model). The owls are some of the most impressive items that we’ve printed. The owl in the picture below isn’t more than 2 inches tall; the detail is good enough that if you look from the right angle, you can see the nostrils clearly.

In general, we’ve not had much trouble with the printer. We’re always tweaking the settings and we have to go through the optimization process every time we try a new filament, but our print quality has been steadily improving. We’ve had the print head jam once; cleaning that out apparently involved a small torch.

This is probably my last update on the M2, as I’ve graduated and I’m not hanging around the lab anymore. If you want to hear more about it and see more pictures, you can take a look at this Google+ album. Also, our advisor is posting some of his designs on thingiverse.

3D printers still aren’t ready for use by the general public. We’ve got ours working rather well, but keep in mind that it’s in a lab full of engineers who like to tinker. High quality printing takes a lot of time and tuning. That said, the process isn’t complex, and the default settings that MakerGear distributes are workable in most situations. I’m interested to see how 3D printing changes as it moves into the mainstream.

CCSVLIB 20130307

We have a MakerGear M2

6 Replies

The research group I work with ordered a MakerGear M2 back in November, and it finally arrived yesterday. My lab-mates and I took the afternoon to pull it out of the box and become acquainted with it. We ordered the pre-assembled package, so this was mostly a plug-and-play operation.

I arrived in the lab shortly after the unboxing, so I can’t describe the packaging in great detail. I’m told that the box was covered in “fragile” labels, and that the zip ties on the printer were color coded; red for ties that should come off and black for those that are permanent. MakerGear uses high-end chocolates are for packing inspection tokens. I can say from experience that those are very good. Before shipping the assembled printer, the folks at MakerGear printed two test patterns, a bracelet and a gorilla head; both were shipped with the printer.

We started by fiddling with the motion on the head and the bed. Before printing anything, we brought the head and the bed up to printing temps (185 C and 60 C respectively, for PLA) and ran some filament to clean the head. Finding the right calibration settings took three or four attempts at printing something. Somewhere in the process of learning how to manipulate the machine, we managed to move the bed to the positive limit in Y and the power/sensor cables snagged on the frame. The power connector simply unplugged. Unfortunately, the sensor wire snapped at the solder connection inside its connector. After half an hour of negotiating with the connector housing, we were able to extract the metal contact and re-solder the connection. One more round of leveling and we began printing a Companion Cube. We’ve apparently got some issues with our configuration for the skirt, and I think we probably need to adjust the head clearance, but our first printing looks pretty reasonable.

I’m certain I’ll have more to say about our printer in the coming days/weeks. It’s an interesting device, and I expect it to be a good toy (or distraction).

jafrro.net

Computers, Marching, and Life