Don't let your Pinball fun be ruined by rebooting problems...
Indiana's Dying
A little while ago, a friend mentioned to me that his pinball machine was dying every time he pressed both flipper buttons. I asked to see this happen in action, and he was kind enough to allow me to operate the game. It behaved very strangely. As soon as both flippers were activated, the machine rebooted. Not being much of a pinball fan, I made some polite noises and moved the conversation on to the real reason I had come to visit, which revolved around picking up a microwave oven. I think what I said at the time was that it was probably the solenoids operating the flippers. My guess at the time was that they were probably drawing too much current from the power supply, causing it to under-volt the central processing unit, which, starved for power, was acting like the machine had just been turned on. In any case, I pointed out that the problem was very likely related to a power supply issue.I Hate Boredom
The microwave turned out to be a dud. As time passed, I often thought about the pinball machine, which was Indiana Jones themed, and wondered if it would ever work properly again. After a while, I contacted my friend and asked how he was doing. Unsurprisingly, he once again asked me about if I could fix his machine. There was nothing on the workbench, so I agreed to give it a try:
My "On Again, Off Again" Relationship With Pinball
The last time I played pinball with any seriousness was in the early 1980's. My family used to go to Old Orchard Beach in Maine for a month over the summer holidays, back in the days when taking a month-long summer holiday didn't raise eyebrows. At the beach, near the pier, there was an arcade that looked like this:
My sister, who was about 10 years old at that time, liked to play video games, especially Pacman and Ms. Pacman. So I would go to the arcade with her to pass the time. Yes, I played with the games, but never really got a charge from them. I still don't play games.
Microsoft Pinball
But I did love the fact that Microsoft included Space Cadet as a free element of its early GUI Windows lineup:
I loved the look and feel of Microsoft Pinball, and always admired Microsoft for hiring Cinematronics to make it. Out of the box pinball in Windows is gone now, a victim of Microsoft's transition to 64-bit, but I think Microsoft Pinball will be forever remembered, considering how many articles have been written about it.
UPDATE: Here's how to download and install Microsoft Pinball on 64-bit
But Why Was Indiana Jones Dying?
As I mentioned at the top, my instincts were telling me that the machine was dying due to an under-supply of electricity to the CPU. Now, WHY the under-supply was happening was a bit of a mystery. Seeing as the event only happened when both flippers were activated, my first guess was that the flippers were simply drawing too much power, and thereby dragging the machine into the netherworld. Considering that flippers are electromechanical devices (basically, a solenoid) that are subject to wear and tear, my guess was that they just needed a service. But that guess was pretty weak. Surely this problem wasn't unique?The Internet: An Ocean of Pinball Related Information
Due to my total lack of background in pinball machines, other than as a (very) occasional user, I decided to do a little bit of research. Thank god for the Internet. The simplest Google search turns up a cornucopia of articles, how-to's and even videos on how to fix pinball machines. There are some very dedicated aficionados producing articles and even videos on my exact problem, albeit featuring different machines than the one I was working on:
In fact, this problem is becoming so common, people are even making a business of fixing it with their own specially crafted add-on components:
Pinball Machines Are Dying Like Flies
I found out something a little troubling while performing my research. The pinball machines that had occupied much of my youth are dying out. When I was growing up, they were everywhere. Now, they're going extinct and becoming collectors items. Besides the fact that everything is going digital these days, pinball machines occupy a lot of space and make a lot of noise. Compounding this is the fact that after X years, their most vulnerable internal electronic components (mostly capacitors and diodes) start to give up the ghost, accelerating their voyage towards "electronics heaven". The situation with pinball machines is very similar to that of vintage solid-state amplifiers like my Yamaha DSP-A3090, which was first released in 1995, and even younger units like my Antique Sound Lab Hurricane, which was released in 2005. Both units have had failures related to component aging - and it's mostly because of the amount of heat they generate. They are literally cooking themselves to death!A Primer On Common Component Failures
When Diodes Go Bad
Normally a very reliable component, diodes go bad when they experience an over-voltage, or "spike". The tolerances for most diodes are very tight these days, and transient voltages can easily damage their delicate junctions.
The following picture features a blown diode in an Atari Asteroids Deluxe machine:
(http://www.aaarpinball.com/AsteroidsDeluxe/AsteroidsDeluxe.htm)
Sometimes diodes are ganged together in a specialized package known as a bridge rectifier. A Bridge Rectifier uses four diodes hooked together in a special way so as to help convert Alternating Current (AC) to Direct Current (DC). They are commonplace in power supply circuits.
Here's a picture of a blown bridge rectifier from a Conrad Johnson PV-7 pre-amplifier:
(http://www.lencoheaven.net/forum/index.php?topic=4279.0)
When Capacitors Go Bad
Capacitors are less reliable than diodes because of how they are constructed. Basically a "swiss roll" of aluminum foil and liquid impregnated paper, they go bad when they dry out.This picture is of a blown capacitor on a computer motherboard:
(https://en.wikipedia.org/wiki/Capacitor_plague)
The Good News and the Bad News
The Good News
The good news is that when it comes to repairing equipment with broken diodes and capacitors, cost is normally not a big consideration. Diodes and capacitors are very inexpensive, widely available and small. They can easily be either picked up at an electronics store or ordered from a very wide range of sources including eBay, Amazon, TaoBao, Mouser, Active. When I was young, before there was an Internet, I would bicycle over to Radio Shack to buy electronics components and books like this from Forrest M. Mims III:
(http://hackaday.com/2017/01/18/forrest-mims-radio-shack-and-the-notebooks-that-launched-a-thousand-careers/)
Sadly, this avenue of learning about electronics is no more. Radio Shack stopped selling this kind of stuff a long time ago. On the positive side, there are now innumerable online sources of electronics tutelage, including Instructables, for whom I have written a couple of articles, including one on making a 3D-Printer from old disc drives, which I worked on with a man named Mark Rogivue. We initially called it the Infinity, then the Curiosity. As a kit, the unit was designed to cost under USD100.00, but the build volume was pretty small, at 3 cubit centimeters:
The Bad News
There's a few pieces of bad news when it comes to doing this stuff:Danger
One of the worst pieces of bad news is that messing around with capacitors (or electronics in general) can be painful or even lethal if you do not know what you are doing - especially if you are working in power supplies. I am not going to even offer a basic electronics safety course as part of this article. I don't want to assume any liability and I don't have the room. Please, if you choose to work with electronics, take some training on how to do it safely, or connect with someone who can show you how to do it safely. You have been warned. Proceed with caution and in the understanding that if you end up hurting yourself, someone else or property you have only yourself to blame.
Time
Fiddling with broken electronics can be very time-consuming. Personally speaking, I do it as a means of relaxation, especially when I am thinking about something else that's complicated. Believe it or not, I often need to be in the process of solving one problem to make any progress on solving some other, usually more difficult, problem. So, I often keep my hands busy on a minor challenge while settling a major challenge. I know this sounds strange, but it's true. If you are like me, you might want to give it a try. The process and sense of accomplishment that comes from fixing something often translates somehow to the bigger stuff.
Failure
I should be honest here and state that not all electronics problems are trivial and not all of them can be fixed easily - or at all. Some electronics is so horribly tiny, it's impossible to work on (almost all mobile phones). Some electronics feature Integrated Circuits and other proprietary or specialized parts that you just can't get a replacement for. Some electronics problems are just beyond your skill or troubleshooting level. Some problems require tools that are too expensive. Be aware that not everything can be repaired within an acceptable cost window. These units are often labeled as being Beyond Economic Repair (BER).
Getting Back to Fixing Indiana Jones
Now that we know what we are in for, let's talk about the Indiana Jones situation. Seeing as it was pretty clearly a Power Supply issue, the question then became one of supply or demand. My initial reaction was demand - I figured the aging solenoids in the flippers were drawing too much power. After doing some research, it became clear to me that it was actually a supply problem. The Power Supply capacitors were very frequently the culprit. After finding a very similar characterization online, I posted the following message:
(https://pinside.com/pinball/forum/topic/machine-powering-down-after-hitting-both-flippers)
In terms of this fix, I owe a large debt to Robin, the founder of www.pinside.com for the following post:
Armed with this information, I started to ask my friend about the Power Supply section of his Indiana Jones machine. He sent me this photo:
Upper Right
On the right side, the section of the board that is of interest is the AC-DC section, where the incoming voltage is converted from Alternating Current (AC) to Direct Current (DC). This is accomplished with a square shaped bridge rectifier and a duo of smoothing capacitors:
Upper Left
On the left side, there's what looks like another bridge rectifier and a single smoothing capacitor:
So it looked like there were five (5) capacitors to replace - but what size? Capacitors act a bit like cups that hold hot water. The volume of the cup is measured in a unit called Farads, after the famous physicist, Michael Faraday. There's one more rating, which is the voltage rating. You can think of it as the temperature of the water. A plastic cup melts if boiling hot water is poured into it. The same thing happens with a capacitor - too much voltage and it will fail. The capacitors we found in the Indiana Jones Pinball machine were rated at 15,000μF @ 35v, which means they were rated to "hold" 15,000 microfarads of power that had been charged up to 35 volts:
So the next task for us was to source five (5) capacitors of equal or better rating. Sourcing electronics is pretty easy. I found the following replacement units on www.taobao.com, for about USD1.25 each:
Disassembly
In preparation, I asked my friend to remove the power board. Pinball power boards have a lot of connectors, all of which need to be carefully tagged and documented before the board can be safely removed.As we moved through the board removal process some very interesting things were revealed. For example, we found some strangely marked plugs, and a damaged connector:
Here's what the cabinet looked like with the power board removed:
Once the board was removed, it looked like this. Our focus areas are highlighted:
For the first time, we were able to do a visible inspection of the back of the board. By the looks of it, the capacitors in question may have already been replaced once before:
Not the best quality soldering work!
This also raises the specter of the capacitors being the wrong size or rating, so I had to go find out whether or not the installed capacitors actually matched the recommended ones. Luckily, I found an online version of the Indiana Jones Pinball Machine Operations Manual.
Inside the manual, I was able to find the schematics for the power circuit:
As well as the loner on the other side of the board:
So now we knew we needed to replace C5, C6, C7, C11, and C30. Consulting the parts list in the Indiana Jones Pinball Machine Operating Manual, the rating of these capacitors is clearly listed:
So the capacitors selected as a replacement for what had been installed would be OK. In fact, more than OK. But a question was begging to be asked. If the capacitors had already been replaced, and were therefore new - why was the machine still resetting? The only way to really find out was to do one of these two things to eliminate them as a source of the problem:
1a) Take all of the capacitors out
1b) Put in a new set of capacitors (cost: USD10.00)
2a) Take all of the old capacitors out
2b) Test all of the old capacitors with an ESR meter (cost: USD50.00), and then probably...
2c) Put in a new set of capacitors (cost: $USD10.00)
What is ESR? How Do We Measure It?
ESR stands for Equivalent Series Resistance. This approach to troubleshooting capacitors adopts the perspective that capacitors act like a resistor under certain conditions, and can be measured as such. With age and use, the equivalent series resistance of a capacitor tends to creep upwards, sometimes to spectacularly high values. This can cause other components in the circuit to malfunction, or even fail. This is especially true of high frequency circuits that rely on resistor-capacitor combinations, or "RC Circuits" (https://en.wikipedia.org/wiki/RC_circuit) which are very common.Under ideal circumstances, a capacitor should offer zero resistance to an electrical circuit, but in reality, that's never true. Capacitors always offer a nominal resistance of some kind, it just depends on the type of capacitor. What an ESR meter does is measure and display the equivalent series resistance of any given capacitor so it can be compared with the nominal value(s) for that capacitor type. If the value is wildly out of tolerances, the part is considered defective. Here's a really inexpensive ESR tester in action:
Why We Bother With Capacitor ESR Testing
In the hobby world, only uber-geeks bother to test out capacitors. Most hobbyists just swap the capacitors out and have done with it. At $1.25 for even a large (15,000uf @ 35v) part, why bother? But in the professional world, knowing exactly what part failed really matters, because clients are paying for the repair and want to know where things went wrong, or the company is paying an in-house tech to do the repair. It also makes sense when you are contemplating changing thousands of units on an assembly line because you may have sourced substandard capacitors, or are dealing with a large recall program as a warranty support measure. In those circumstances, rooting out the exact cause of the problem makes sense. Here are some common ESR values:
(http://peakelec.co.uk/downloads/esrguide-en.pdf)
Six Sigma Isn't Always Your Friend
In the hobbyist world, changing out all of the capacitors makes sense because of the way things are made. Usually, all of the capacitors in a given piece of electronics were purchased at more or less the same time - so they are going to fail at more or less the same time. This is especially true in an age of universal quality control systems, where deviations from the norm are not tolerated. The application of ISO type process controls in manufacturing creates the potential for "clustered" failures because variation is kept to a minimum. Six sigma quality means your capacitors are likely to fail around each other, because they are all basically "clones":
(http://lablean.blogspot.hk/2015/02/six-sigma.html)
The Replacement Capacitors
The ESR Meter
Prepping for Surgery
With the board out of the machine, and the operating theatre ready to go, all we needed to swap out the capacitors were the following things:Soldering Iron
Solder Sucker
Solder
No comments:
Post a Comment