Tag Archives: Toyota

Robots and the Law

Posted on by

In the April issue of PCD&F/CIRCUITS ASSEMBLY, I wrote about the need for a balance between autonomous machinery and human-operation equipment. I wrote the piece in the aftermath of the Malaysia Airlines Flight 370 disappearance, and referenced, among other things, the Toyota sudden unintended acceleration problems and the self-driving cars that are beginning to appear on US streets.

Seems I’m not the only one working their way through this. On May 5, a pair of researchers at the Brookings Institution began a series of papers (The Robots Are Coming: The Project On Civilian Robotics) that considers the legal ramifications of driverless cars.

That led me to Google, which uncovered a few more references to potential tort roadblocks.

While my work considered the technical and emotional issues that always factor into to any major technology shift, the legal aspects are equally in play here. For those interested in the subject, the Brookings Institution project is especially worth a read.

 

 

 

 

Tin Bells Going Off

Posted on by

You may remember that more than a year ago there was much speculation that tin whiskers may be behind the Toyota unintended acceleration problem. At the time I spoke out because there was no data to support the speculation. Now there are data, as Mike Pecht and his CALCE Team at the University of Maryland have found numerous tin whiskers in the Toyota brake assemblies of concern.

Although the tin whiskers were not implicated in any failure, their presence is cause for alarm, and action should be taken to address this issue. Tin whiskers should not be found in mission critical devices. Pecht’s team has an algorithm that calculates the risk from tin whiskers that are discovered. The risk is 140 per 1 million — not high, but with a million or so Toyotas on the road, clearly this is cause for alarm.

As you may know, I live in Woodstock, Vermont.  Many friends have asked how we are doing after hurricane Irene. Personally, my wife and I escaped with no damage to our house and only a bit of inconvenience (no water for 5 days). The town of Woodstock suffered considerable damage, but was, on the whole, fortunate. Some of the neighboring towns had all roads in and out washed away. Route 4 between Woodstock and Rutland has numerous sections destroyed. The flooding was declared by the governor to be the worst disaster in Vermont history. The photo is from the Valley News. It shows a wooden pedestrian bridge built to carry supplies into Bethel, VT by foot. There is no passable road, even for ATVs.

Best,

Dr. Ron

 

Tin Liability: Careless Whiskers and Toyota Acceleration

Posted on by

A failure mode is reemerging that has been responsible for the loss of billions of dollars worth of satellites, missiles and other equipment — the culprit is the electrically conductive entities known as ‘tin whiskers’.  Now one research group says that tin whiskers may be responsible for the sudden acceleration in Toyota Camry models from the year 2002 and possibly beyond.

Earlier this year we reported that the US Department of Transportation (DOT) said that Toyota’s problem was not in electronics.

Now, University of Maryland’s Center for Advanced Life Cycle Engineering or CALCE researchers have found the potential for tin whiskers in the electronic control module or ECM.  Circuits Assembly broke the story, quoting the CALCE report as follows:

“The ECM contains surface mount electronic devices connected with tin-lead solder to a multilayer PCB. … Interconnect terminals of the perimeter leaded devices were found to be plated with tin. In addition, tin plating was found on terminal pins of the edge connections. As previously discussed, tin-finished leads can grow tin whiskers which can lead to unintended electrical shorts.”

“We know whiskers can form on tin finished terminals,” said Michael Osterman, senior research scientist and director of the CALCE Electronic Products and System Consortium. said.  “In this case, Toyota has tin plating in a rather sensitive area, where the system relies on changes in resistance to provide a signal for acceleration.”

The studied pedals furthermore have been shown to cause shorts known to spur sudden unintended acceleration.

The odds of tin whiskers: 140/million. Someone known to this blogger recently drove a 2010 Camry and noticed subtle but perceptible decelerations that were not led by the driver. Was it tin whiskering?  Hard to say, even CALCE’s study figures that the whiskers would only form in 140 cars per million, which is statistically very significant but als makes it statistically unlikely that my friend’s only Camry experience would be on the wrong side of those odds.

It’s also worth noting that the whisker syndrome is probably not limited to Toyotas.  Nonetheless, the spotlight has fallen where it has fallen, and tin whiskers pose a serious problem in that warrants attention.

Tin whiskers. Tin whiskers develop — or may develop — on any product type that uses lead-free pure tin coatings.  Thus, in greener, lead-free products, tin whiskers can pose a major safety, reliability and potential liability threats to all makers and users of high reliability electronics and associated hardware. The CALCE brain trust concluded that existing approaches are not sufficient to control tin whiskering in high-reliability systems such as automobile electrical systems.

US Secretary of Transportation said Toyota is “all clear” in February. The official blog of the US Secretary of Transportation on February 8, 2011 stated:

NASA engineers pored over more than 280,000 lines of software code looking for potential flaws that could initiate an unintended acceleration incident. Alongside NHTSA, they bombarded vehicles with electromagnetic radiation to see whether it could make electronics systems cause the cars they control to gain speed.

And today, their verdict is in. There is no electronic cause behind dangerous unintended acceleration incidents in Toyotas.

To read more about it: http://supply-chain-data-mgmt.blogspot.com/2011/02/us-dot-says-toyota-problem-was-not-in.html

We will continue to follow this story.

Tin Whiskers and Toyota: Collision Course?

Posted on by

New criticism of the reports by the National Highway Traffic Safety Administration and NASA Engineering and Safety Center that led the US Transportation Secretary to publicly absolve Toyota of unintended acceleration problems in its vehicles is breathing new life in what the mainstream media had decided was a closed story.

When the US agencies released their reports in February, Sec. Ray LaHood stated that the findings by the NHTSA and NASA proved Toyota’s electronics were not guilty of causing unintended acceleration. “The verdict is in,” LaHood said. “There is no electronic-based cause for unintended, high-speed acceleration in Toyotas.”

Not so fast, said Safety Research & Strategies, which this week went to press with a report condemning the earlier findings for everything from flawed analysis to conflict of interests.

In the report, SRS claims the tin whiskers found in the vehicle samples provided to NASA did in fact reveal a failure mechanism that was ignored in the NHTSA report, yet that mechanism in accelerator pedal sensor circuits can cause resistive shorts that could lead to acceleration.

The report has become a hot topic among a group of printed circuit board reliability experts, who are pointing to the “extremely small sample size” of vehicles used by NASA to perform its investigations. “There are millions of Toyotas on the road today but NASA was able to look at only a handful,” wrote Bob Landman of HRL Laboratories, on the IPC TechNet Listserv. “Despite the small sample size, they found whiskers.  The Law of Errors tells you what about this fact?  That whiskers are a significant finding.”

Landman noted that in one case, NASA found whiskers in a pedal assembly after a woman who had an incident of sudden acceleration was provided the defective assembly by the dealer that fixed her car. “She learned of the [Department of Transportation] investigation and gave them the assembly, and it found its way to NASA where [researchers] found whiskers shorting the leads of the potentiometer.

Landman also said NASA demonstrated a braking problem under a test track sudden acceleration simulation.  “A NASA driver was strapped in, a NASA passenger had two switches, one to cause sudden acceleration at 45 mph and the other to safely turn off the the sudden acceleration so the vehicle could be brought to a stop.  What happened?  When sudden acceleration was initiated, the throttle was at 100% so there was no vacuum assist and the driver, using both feet on the brake pedal, could not stop the vehicle! It was found that it would take 600 pounds of brake force on the pedal to cause the brake to slow down the vehicle. Clearly, the software does not allow the brake to override the pedal. This is a defective design.”

“Something is rotten in this [NHTSA] report, it seems to me, and SRS found it,” Landman said.

Has Toyota Solved SUA?

Posted on by

I remain skeptical in the light of Toyota’s latest statements that driver error caused “virtually all” of the unintended acceleration problems that have plagued the carmaker for the past decade.

Per the Wall Street Journal, NHTSA analysis of the affected cars’ “black boxes” found instances in which throttles were open and brakes hadn’t been deployed, suggesting drivers were pressing the gas, not the brake. (NHTSA isn’t commenting.)

The dreaded sudden unintended acceleration is supposed to have caused up to 89 deaths in 71 crashes since 2000.

Admitting in advance that my reasons are somewhat circumstantial, I would counter that Toyota

1. Has failed to provide solid evidence to refute a university researcher’s claim that the electronics wiring could be the cause, and that the car’s software lacked a fault code to point out the defect.

2. Has failed to explain why the rate of accidents attributed to SUA is not similar in competitors’ vehicles.

That said, the shims the carmaker has installed beneath the accelerators, the complaints over SUA appear to have subsided. Perhaps Toyota was correct, after all.

(Full disclosure: My wife drives a Prius.)

Sans Data, Toyota ‘Whiskers’ Claims Razor Thin

Posted on by

After my recent post on the fact that no data link tin whiskers to the Toyota sudden acceleration issues, there continue to be more posts saying things like “Tin Whiskers Implicated in Unintended Acceleration Problems.”

Many of these posts link back to the earlier TechEye post. The basis for all of the posts is a paper written by EurIng Keith Armstrong titled, “Toyota ‘Sticking Pedals’ Recall is a Smokescreen,” and subtitled, “Their sudden unintended acceleration problem is caused by electronics either due to EMI, lead-free soldering or software ‘bugs.’ ” It does not appear that Armstrong’s paper was sponsored or refereed.

Since it appears that this entire wave of reporting implicating tin whiskers, in this important issue, emanates from Armstrong’s paper, it is helpful to quote his entire comments on tin whiskers

9.0 Lead-free soldering:
In recent years, various countries and trade blocs (including the European Union) have banned the use of lead on electrical solder, on the basis that lead going into landfill when electrical and electronic products are disposed of is bad for the environment, and hence for people.

But many accuse them of being shortsighted -– lead has been added to solder in quite large amounts for many decades because it made the other main constituent, tin, behave much better, considerably improving reliability.

Now that lead has been removed from solder, which is now mainly tin (with a little silver and copper added) all sorts of new possibilities arise for short-circuits and open-circuits, and intermittent shorts and opens, mainly on printed circuit boards (PCBs) and mainly associated with small-footprint integrated circuits (ICs), especially ball-grid arrays (BGAs).

Its really just another cause of intermittent or fixed short-or-open circuits in electronic PCBs and modules – but one that would not have been any problem until a few years ago, and so could have caught Toyota by surprise.

John R Barnes has created a monumentally huge library of references to the problems of lead-free soldering, especially tin whiskering, see www.dbicorporation.com/rohsbib.htm. Prepare to be totally overwhelmed!

Removing lead from solder has the following effects:

9.1 Tin whiskers
These will grow out of soldered joints and can contact other conductors, causing short-circuits between PCB copper traces and the pins of connectors. They are often no longer than 0.5mm (about 1/50th of an inch) but can grow to 1mm (about 1/24th of an inch) or longer, especially in damp conditions.
Even at 1/50th of an inch they can short between the pins on a modern integrated circuit (IC). And the process of removing the PCB for inspection can brush them off, so you never find them.
And if you didn’t accidentally brush them off, they are so thin they are very hard to see – you need a powerful microscope. They are as fine as the finest spider-web threads, yet can carry sufficient current to short-out the electronics. You won’t see them unless you are looking for them.
Being so thin, they can wave around in the breeze and/or due to shocks, vibration and acceleration, causing intermittent short-circuits.

The iNEMI organization has published guidelines (www.inemi.org) on how to ensure that tin whiskers don’t grow too long, but I don’t know to what extent these are followed by suppliers of electronics to the car industry in general, or Toyota in particular.

(Photo courtesy NASA)

Note that, in this paper, there are no data or any evidence regarding tin whiskers discussed from investigating any of the vehicles in question. All of this paper is opinion. In addition, the title of Armstrong’s paper leaves no room for any other cause: it has to be electronics or software. This position is very strong indeed for having no supporting data.

More recently Bob Landman added these comments to the tin whisker discussion: “The increased use of electronics in automobiles when mixed with RoHS can make for a deadly cocktail. We don’t know what the causative agent [in regard to the Toyota recalls] was, but I have heard recently of new autos showing up at dealers that will not start. That cause has been linked to tin whiskers.”

Bob heard this. There is no report and no data. Until Bob gives us a reference for some analysis and data, his comments are little more than hearsay. I searched the web in vain to find information related to Bob’s quote. In addition this comment is a little surprising, tin whiskers are usually associated with a certain amount of aging, hence not usually found in new products.

That tin whiskers exist and cause failures is irrefutable. NASA has an excellent website related to tin whiskers and failures caused by them. However, the total number of tin whisker fails reported is less than 100. Many other types of electronic failure modes would appear to be much more common.

My purpose of writing this post is not to suggest that tin whiskers are not a concern in lead-free electronics. However, it is a fundamental principle in engineering and science to only make pronouncements on how something failed, when they can be supported with data. No data support implicating tin whiskers in the Toyota incidents. It is also troubling how readily many people referenced the work of Armstrong without apparently reading what he said and checking his sources and lack of data.

Cheers,
Dr. Ron

Mysteries of Engineering

Posted on by

I (and many, many of us, presumably) have been reading more about all of the Toyota woes and the to-date unanswerable questions. Still, so much of the material written about the issues seems to be coming from the untrained. Certainly, human behavior suggests that some of these problems could be the result of operator error. But, I’m not an expert in human behavior, so I can’t really say. And, certainly, problems do crop up in complex machinery, like cars. I don’t know if that supposition falls within my area of expertise, but a few decades of operating motor vehicles gives me some personal empirical data on that one.

The area that does bother me the most is probably those that speculate that since the problem hasn’t been found, it doesn’t exist. This is an area where I can claim some level of expertise as well as plenty of personal empirical data.

It is possible to spend uncountable hours testing various possible conditions and still never uncover the one scenario that will cause a systems failure in the hands of the general public. Many years ago, I worked for a company that designed, built and sold projectors. In that day, these were big things with short-life, very hot, incandescent lamps. We thought that we had done a very through job of testing under various conditions and had been selling the product for a little while when reports started filing in of bulbs exploding. It wasn’t just a simple break. The bulbs were exploding with such force that the bulb area was filled with a fine grained, razor sharp glass dust. Nasty.

During a weekend burn in session with a couple dozen projectors, including some returned from the field, the engineer monitoring the process thought he heard a gunshot and dove to the floor. It wasn’t a gunshot, but it was the first clue in a long investigative process that did end up finding the problem. It seemed that if a bulb was too deeply seated in the socket by a couple of millimeters, the reflection of the filament in the mirror would exactly line up with the actual filament, causing it to melt and arc. The arc would run in one direction, down the filament leg to the base and stop.

One filament leg had a few coils of small diameter tungsten wire wrapped around it. The other leg did not. Depending on the orientation of the supposedly non-polar bulb, the arc would either run down the leg with no coil or the leg with the coil.

If the arc ran down the leg without the coil, nothing happened other then the bulb needed to be replaced. If it ran down the leg with the coil, that small amount of additional vaporized tungsten increased the internal pressure sufficiently to explode the quartz bulb in a very catastrophic manner. Okay, now that’s weird and obscure. Technically, you could call it operator error. If the customer had just inserted the replacement bulb the exact same way we inserted the bulbs during production, the problem would never have happened. But, realistically, it was a design flaw that set the customers up for a failure.

Duane Benson
Duck and cover

http://blog.screamingcircuits.com/

Toyota is as Toyota Does

Posted on by

Everyone else seems to be writing about Toyota sudden acceleration problems, so I should probably do that too.

Or should I? Personally, I have absolutely no solid information about what’s going on with Toyota cars. There’s an awful lot written, much of if by people that also don’t have any real information on the subject. Here’s what I do know though:

  • Some people (some with actual knowledge and some without) are speculating that electronics might have something to do with the problems.

That’s about all I know relative to the specific concerns. On the soft side, I do know that people tend to pick on the big guy. Funny how none of this was big news until Toyota became the #1 car maker in the world. Coincidence? Maybe. Maybe not. I also know that in any system there are gobs of places where issues can lead to failures. Of course, to counter that, I know that good, well thought out design – both in the hardware and the software, can produce a quality product that will keep working. Of course, to counter that, I know that good, well thought out design – both in the hardware and the software, can produce a quality product that will keep working. In summary, I really don’t know anything about the Toyota issues

However, any time some sort of actual or potential technical problem makes big news, it’s not a bad idea for those that design and build things to take a step back and evaluate our design practices. I’ve got software in my past, so I’d have to suggest a good solid code review, if you don’t already do one, but today, I’m talking about hardware so I’ll sample just a few things to double check.     

* Those pesky land patterns: Does the land pattern fit the part? Will the copper area and stencil opening allow for a good solid IPC-passing solder joint? It’s so common (as you well know if you read here regularly) to re use or create new CAD part foot prints. Make sure the foot print, stencil, mask and silk layers fit properly.

* Vias in pads: Plug them and plate over them when using small parts. If the solder surface is big enough, like with a power component, you might be able to just cap them, but don’t leave the vias open. In some cases, you may be able to leave very tiny vias open on thermal pads, but it’s best never to.

* Thermal mass: This is important both for operation and for assembly. If you’ve got components that sink and/or generate lots of heat, make sure there is enough air flow to cool them during operation and make sure that the assembly house can build it. Put a couple of high thermal mass parts too close together and an otherwise perfect PCB assembly may end up with some cold solder joints or damaged components that later come back to bite you or your customers.

There are lots of other things to check out too, but those three are just some of the more common traps to keep tabs on.

Duane Benson

I don’t have a Toy Yoda. If I did, I’d sell in on eBay.

http://blog.screamingcircuits.com/

Missing Code in Toyota Claim

Posted on by

Toyota today claimed Prof. David Gilbert’s testimony on the sudden unintended acceleration isn’t representative of real world situations.

However – and this is important – Toyota makes no mention (at least in this report) about Gilbert’s more important finding: that Toyota’s on-board computers contain no defect code for the problem, which speaks to the reason the company’s diagnosis is (according to several experts) incorrect.

Where’s There’s Toyota, There’s Fire

Posted on by

The sudden unintended acceleration problems in Toyota’s vehicles have touched off a firestorm of controversy over the cause(s). Now, a professor of automotive technology at Southern Illinois University has entered the fray, testifying before Congress that the trouble locating the problem’s source could stem from a missing defect code in the affected fleet’s diagnostic computer.

In testimony before a house subcommittee  Tuesday, David W. Gilbert, a Ph.D. with almost 30 years experience in automotive diagnostics and troubleshooting, said his initial investigation has found problems with the “integrity and consistency” of Toyota’s electronic control modules to detect potential throttle malfunctions.

Specifically, Prof. Gilbert disputed the notion that every defect would necessarily have an associated code. The “absence of a stored diagnostic trouble code in the vehicle’s computer is no guarantee that a problem does not exist.”

In fact, using a 2010 Toyota Tundra, Prof. Gilbert discovered electrical circuit faults could indeed be introduced into the electronic throttle control system without setting a diagnostic trouble code. “Without a diagnostic trouble code set, the vehicle computer will not logically enter into a fail-safe mode of operation. … Since the vehicle computer will only react to defective sensor inputs outside of the range of programmed limitations if the circuit is not defective; it must be good.” In other words, because a code did not exist for the sensor to inform the on-board computer of a problem, when a short occurred the computer did not recognize the problem, and therefore it took no steps to mitigate it. And absent the code, no defect was entered into the database for post-incident tracking.

Prof. Gilbert further determined that electronic control module malfunction detection strategies were not sufficient to
identify all types of fundamental APP sensor and/or circuit malfunctions. “Some types of electronic throttle control circuit malfunctions were detectable by the ECM, and some were not,” he testified. “Most importantly, the Toyota detection strategies were unable to identify malfunctions of the APP sensor signal inputs to the ECM.” (Watch this video of Dr Griffin’s test at his university test track.)

Yikes! If Prof. Gilbert is correct, this could explain why Toyota engineers have failed to diagnose the electronics as a potential source of sudden unintended acceleration. As one reliability expert told me, this could be the smoking gun.

We await Toyota’s response to this revelation.