What Does the Berkeley ITS Ridership Report Actually Say?
There’s a disturbing trend in recent “official” reports on the high speed rail project. Authored by people who either do not understand HSR or who have shown opposition to HSR, these reports take known uncertainties, turn them into controversies, and slap an indefensible but high-profile, overstated conclusion about how their findings suggest huge problems with the HSR project.
We saw this with the Legislative Analyst report, and definitely with the State Auditor’s report.
Now it seems the Berkeley Institute for Transportation Studies’ report on HSR ridership has fallen into the same trap. If you actually read through the entire document, you see that it primarily consists of a long and in-depth debate between the report’s authors and Cambridge Systematics over the finer points of ridership modeling theory. In that sense it is a very typical academic document, where two groups with different views on theories battle it out in technical reports.
Despite the fact that the ITS authors generally commend CS on their modeling, and focus their criticisms on specific aspects of the modeling that they disagree with, the ITS report jumps to its own unsupportable conclusion:
it is not possible to predict whether the proposed high-speed rail system will experience healthy profits or severe revenue shortfalls
The problem is that the ITS report makes the same error they blame CS for making in the study – reaching a conclusion that isn’t supported by the evidence. As with the State Auditor’s report, the CHSRA is also crying foul on this conclusion. CEO Roelof van Ark wrote in a letter included in the report package that “this is an extraordinary statement for which we find no foundation in the Draft Report.”
Further, the report appears to suffer from another common error in official assessments of the HSR project – a belief that the status quo in California will last indefinitely, and that the HSR project’s claims that it will produce significant shifts in traveler behavior should be greeted with skepticism. There is a mountain of evidence that suggests ridership will quickly shift to bullet trains, as in Spain and Taiwan. But the ITS folks don’t seem to consider this as a possibility.
Part of the problem is that the ITS authors are not unbiased observers. The lead author for ITS, Samer Madanat, also authored a extremely flawed and misleading anti-HSR report last fall. In fact, some of the same errors Madanat made in that report crop up here, and are used by the ITS authors to claim that the HSR ridership study is flawed.
Let’s take a closer look at the report. ITS recognizes that the study generally follows accepted principles:
We are, for the most part, satisfied with their responses and agree that their work on this project meets generally accepted standards for travel demand modeling. We are, however, concerned about the impact of some of Cambridge Systematics’ modeling decisions on the reliability of the forecasts based upon these models.
In other words, the ITS authors did not find any evidence that disproves the HSR ridership study. Instead, they took issue with some of the modeling choices made. It’s not matter of them finding clear errors in the ridership study, but of a difference of opinion among experts on how the study should have been conducted. That’s an important distinction that is likely to be lost in the media reporting on the study, and a distinction likely to be ignored by project opponents.
One of the criticisms ITS makes of the study is on how inter-region business travel is handled:
On Page 3-2 of the “model system development” report, the market segments are defined. According to this definition, all business or commuting trips were assigned to peak conditions. This is potentially a serious problem.
Cambridge Systematics’ response to this comment is that they have followed common practice in regional travel demand modeling. While we don’t doubt that this is the case, this fact is irrelevant to the subject, which is an interregional travel demand model. For interregional travel, quite a few business trips are made in the off-peak periods. For example, the California Travel survey shows over 25% of business trips in the off-peak periods.
The effect of this assignment is that a measurement error is added to the level of service attributes for off-peak business travelers. Even if this measurement error is totally random it will lead to biased and inconsistent parameter estimates. This measurement error is in fact not random – off-peak business travelers face better levels of service for auto and worse levels of service for other modes.
In fact, the “serious error” is on the part of ITS. They are assuming here, quite openly and blatantly, that HSR travelers will behave exactly like existing travelers.
There is plenty of reason to believe that’s not how it will work. Business travelers currently travel off-peak for airlines because of the travel time involved, because last-minute fares tend to be cheaper for off-peak travel, and because getting to and from airports is difficult at peak times due to traffic.
HSR would provide qualitatively different choices. If your destination is downtown LA and you’re leaving from the Bay Area, a peak trip becomes quite plausible. If you live in, say, Oakland, you can take BART to Transbay Terminal or SJ Diridon and hop on an HSR train and be in downtown LA by 9 or 10 in the morning, and make your return trip around 4 or 5 PM, getting home around 6 or 7 depending on your final destination.
The ITS authors need to defend their assertion that HSR travelers will behave exactly the same as air travelers. They can’t just assume that’s the case. We need to see evidence it’s the case before we can accept this criticism.
Here’s Cambridge Systematics’ response on the peak travel issue:
The ITS Draft Report claims that the use of peak period service levels for all business/commute travel is “potentially a serious problem” for an interregional travel demand model. We disagree with this conclusion. In fact, this approach is reasonable and acceptable for both regional and interregional modeling.
The reviewers acknowledge that it is standard practice in model development to assign business and commute trips during the peak period. This convention simply reflects the fact that the majority of business travelers and commuters travel during the peak period and face peak-period levels of service. This pattern holds both for regional and interregional travel as the reviewers note. Therefore, using peak-period travel impedances for the business and commute travel market is a reasonable approach for a planning-level model for interregional travel.
The ITS report also repeats a flawed statement made by Madanat in the fall 2009 study he authored. To refresh your memory, here’s what he said about travel time:
Travelers heading to Los Angeles from San Francisco, for example, will consider the time it takes to go to and from airports at each end of the trip, versus the time spent getting to a high-speed rail station. Time spent on the line-haul portion of the trip (actual flying or riding time) is more productive than the access and egress portions. But if access and egress times from HSR stations are as long and onerous as those for air, passengers will save time by driving to an airport instead.
“High-speed rail trades unproductive access and egress time for productive line-haul time,” explained Madanat. That is advantageous to travelers, and they are willing to spend an extra hour or more in line-haul time if egress and access time are diminished. Air travel between some cities in Japan has become nonexistent, thanks to the ease of traveling by high-speed rail.
Here was my reply: I’m sorry, but Madanat is just plain wrong here. The unproductive access and egress time belongs entirely to airplanes, at least in California. He does not appear to have included the ridiculous security theater involved in air travel that adds up to a half hour to travel times. TSA recommends people arrive two hours before a domestic flight. Add in the travel to LA-area airports, none of which have good mass transit connections (whereas LAUS is the hub of the entire Southern California mass transit network), and it is not conceivable to me that HSR is at a disadvantage in terms of travel times. If anything it is likely to have an advantage, or would be comparable, which is all it really needs to be.
Again, we can look at reality to demonstrate the point: if HSR was such a bad deal, why does the Acela have half the market share on the Northeast Corridor? Madanat apparently didn’t speak to actual Acela users:
Barry Ginsberg of Deer Park, N.Y., boarded an Acela train after a meeting in Washington.
“It’s a lot less hassle and more comfortable,” Ginsberg says. “When you figure how much in advance you have to get to the airport, it’s a lot more convenient.”
Note that Ginsberg is a business traveler. Anyhow, Madanat’s team makes a similar claim in the ITS report, this one about headways:
Thus, in the final models, the headway coefficient was constrained by the modelers to be equal to the in-vehicle travel time.
The modelers’ expectation would be reasonable if this was an urban travel demand model, but it is incorrect in the present context. This is because, in inter-city travel where headways are longer, passengers do not arrive randomly at the stations or airports, but rather according to the vehicles’ schedule, which implies that the waiting time cannot be assumed to be equal to half the headway. There is therefore no reason to expect the coefficients on travel time and waiting time to be equal. Indeed, a recent study by Adler, for example, concluded that air travelers place 4-5 times as much weight on travel time as on the time difference between their ideal departure time and when service is available.
It has been argued that if service headways are sufficiently low, high speed rail travelers may indeed use the system in a manner similar to some urban transit riders, arriving at stations randomly and waiting for the next trains. For such travelers, constraining the waiting time coefficient to equal that for travel time may be appropriate. It is clearly inappropriate for air travelers, however.
This is merely an assertion by Madanat that is unproven. He’s basically saying that if headways are low, people won’t just show up at a station and board a train, they’ll schedule around when the HSR trains are operating. But by 2035 there will likely be significant demand for HSR trains, and headways could very well be frequent enough that it would be feasible to essentially drop by the station and grab the next available train.
In fact, HSR travelers can be expected to behave differently than air travelers – because they already are behaving differently. See the above example about Madanat’s line-haul times: an interview with an Acela rider showed Madanat didn’t know what he was talking about in terms of how intercity travelers use HSR and understand its differences from air travel. In fact, there is some indication that people living on the NEC already use Acela in the form of an urban system, dropping in at a station to grab the next available train.
Here’s Cambridge Systematics’ response on this issue:
The reviewers contend that constraining this coefficient led to “bias in the model forecast” because the headways for interregional service are much longer than for urban travel, resulting in different arrival patterns for travelers at HSR stations compared to urban transit systems. We disagree with the assertion that planned headways for California HSR are substantially different than for urban rail service. Accordingly, we believe that the treatment of sensitivity to wait times and headway is reasonable and does not introduce any biases.
The frequency of air and rail service has an impact on the time that travelers expect to wait at a terminal or a station, and on the convenience with which they are able to travel close to their desired departure time. Therefore, two separate components are used to reflect the impact of service frequency on travelers’ choice behavior.
Based on observed data and expert input, average wait times of 55 minutes were established for air travelers and 15 minutes for HSR and rail travelers. Similarly, separate terminal processing times of 18 to 24 minutes were established for air travelers and 3 to 12 minutes for HSR and rail travelers. The sensitivity to wait and terminal time is twice as high as the sensitivity to travel time, consistent with the literature and practice.Beyond this traditional “wait-time” component, the sensitivity to headways was introduced as an additional component to reflect travelers’ anticipated reaction to schedule convenience.
The proposed HSR service offers a new paradigm of interregional service. The proposed HSR headways are more comparable to the best urban rail services in the U.S. rather than current intercity air and passenger rail services. In this context, the value that was used for the headway coefficient was debated during the model estimation and validation process, and a value consistent with urban rail systems was determined to be appropriate given the planned frequencies of the California HSR system.
During calibration of the original model, there was an overestimation of air trips in markets with low frequency of air service and an underestimation of air trips in markets with high- frequency air service.
The merits of different potential interpretations and values for the headway coefficient were documented and discussed during the peer review process. The con- straint on the coefficient was deemed to be a more reasonable approach than introducing higher alternative-specific constants that would have a greater impact on model sensitivity.Finally, it also should be noted that the short headways and corresponding short wait times account for a small portion of the interregional air and high-speed rail travel times in this study. As a result, the impact of using different assumptions on coefficient values will be correspondingly small.
In other words, Madanat is in error in assuming that HSR travelers will behave like air travelers. I agree with Cambridge Systematics on this point.
Another claim ITS made was about airport choice:
The modeling approach does not explicitly consider the choice of airport or station. Given the trip origin and destination, the model determines the airport or stations that would be used for access and egress “taking into account the level of service of the access and egress modes and the frequency of service at each station and airport ….” (See Appendix A). Such “all-or-nothing” assignment is behaviorally unrealistic since, depending on their desired travel schedule, access/egress modes and other factors, travelers may choose different stations.
The failure to consider station choice is important when comparing ridership for the Altamont and Pacheco corridors. In the Altamont alternative, trips between South Bay locations and Southern California must be exclusively assigned to either a station on the San Jose or the San Francisco lines, and the frequency of service will be accordingly reduced. In reality, travelers in these markets could choose either line, depending on which is most convenient for their particular travel schedule. Failure to consider this possibility means that the inconvenience of the split schedule assumed for the Altamont alternative is exaggerated. The problem is compounded by the inflated value of the headway coefficient, as pointed out earlier. Correcting this deficiency would reduce, although probably not eliminate, the projected ridership difference between the two alignments.
As you can tell, this is intended to sow fear, uncertainty and doubt about the Pacheco choice. But Cambridge Systematics reacted very strongly against this point:
The ITS Draft Report contends that the adopted process for assigning travelers to individual airports and rail stations is “behaviorally unrealistic”. The reviewers contend that the absence of a more elaborate modeling structure for airport/station choice “has a substantive impact on the comparisons of ridership for the Altamont and Pacheco corridors.” We disagree with both points. Further, we believe that a more elaborate “airport/station choice model” is not critical for meeting the objectives of the model development and application work that has been conducted, nor for accurately distinguishing the ridership and revenue potential between the Altamont and Pacheco corridors.
The model currently uses a network-based method that assigns an airport or rail station to all travelers originating from a specific zone. The rule that is used is based on evaluating paths from each origin zone to alternative airports and rail stations. The attractiveness of each path reflects the access modes that are available, the level of access service they offer, and the frequency of air and rail service available at each airport and rail station.
An airport/station-choice model would allow the allocation of a proportion of travelers to different nearby airports and rail stations. However, such an approach would have, at most, a minimal effect on Altamont’s ridership, and then only for a few zones in the study area.
Cambridge Systematics then goes on to provide an example from the SF Bay Area, showing that the only part of the region where there’s any real uncertainty over how riders would behave is in the Menlo Park to Sunnyvale region. The full explanation is on pages 51-52 of the report PDF, and is worth reading to see why the ITS report is flawed to make this criticism.
There are other points where ITS and CS disagree on the details. And that’s typical of an academic discussion where smart people have different interpretations of the same data and how it should be analyzed. God knows I’ve sat through my share of these discussions in my graduate studies in US history, where there are very strong debates and disputes over how to use historical sources.
So it’s important to understand what this report means – and what it does not mean. It does not say that the HSR ridership numbers are proven to be incorrect. What ITS is saying is that their view of proper modeling indicates that the numbers might or might not be right. But CS has their own view on this, backed up by other observers and econometrists, who say the numbers are valid. This is not a case of someone disproving something, it’s a case of different views on how modeling should be done.
Of course, the real question here is the political reaction. We know the media is going to report on this the same way that they reported on the State Auditor’s report, despite its flaws – as a sign that the HSR project is in trouble. The media, unfortunately, is used to framing government as flawed and error-prone. They’re not so used to questioning the assumptions of those that make such criticisms.
It’s always been my view that the state legislature will order new ridership studies. That could be a blessing for HSR backers, since the current ridership study uses surveys conducted in 2005. A survey conducted in 2010 or 2011, after the rise in oil prices and the new, higher level of passenger rail ridership around the state, would probably show higher projected ridership.
Whether that means a delay for the HSR project or not is an open question. Surely we can expect the HSR opponents to demand new EIRs to be conducted. But that doesn’t seem necessary, since the EIRs were certified for a high level of ridership. A lower level means less environmental impact within the same ROW.
Ultimately, this report shows the desperate need to build up knowledge of how HSR works around the world – and across the country – in our state institutions, from the LAO to the State Auditor to our colleges and universities. Most transportation studies experts are too deeply locked into a view that suggests the 20th century will last forever. As we know, that’s not the case. It’s time for these experts to catch up, and stop writing flawed reports.
How The Media Gets High Speed Rail Wrong
There were two important high speed rail reports released this week in California. One of them, by CALPIRG, showed how HSR is a clear success around the world, having no trouble meeting its ridership goals and improving transportation. The other, by the Berkeley Institute for Transportation Studies, questioned some of the assumptions of the HSR ridership study.
Guess which one got covered breathlessly by the media – and guess which one was ignored.
If you guessed “the study that questioned HSR ridership assumptions” as the one that got breathless coverage, with the pro-HSR study being ignored, you win! You may collect your prize, a $6 gallon of gas, at some point in the near future.
The media’s reaction to these two reports reveals how deep the anti-HSR bias runs in the California media. Repeating a trend we’ve seen nationwide, the media here prefers not to provide accurate reporting on an event or, in this case a report – they instead tell readers how that event or report fits into a set of preconceived notions. Specifically, many in the media are using the ITS report to bolster their own belief that government always screws up, that people never ride trains, and that HSR is at best a suspicious and dubious idea that is likely to become a boondoggle.
In contrast, the near silence about the CALPIRG report shows that the media really has no concept of how HSR works. Sure, you get the occasional “HSR is a huge success” article, but it’s usually relegated to the Travel section. The San Francisco Chronicle is a notable exception to this, having sent their main transportation reporter, Michael Cabanatuan, to Japan so he could write a very informative article on HSR.
Most others in the CA media therefore see HSR as something that is foreign, uncertain, and not all that likely to succeed. Even though the ITS report merely said that they disagree with some of the methods used in the HSR ridership study, and even though they did not say that HSR definitely won’t attract enough riders, the media is making it sound as if they did. Because the media doesn’t really understand HSR, they’re selling the public a distorted view of what the study said, and therefore, what is actually going on with the project.
Let’s take a look at some of the media coverage of the study, and explain how they’re getting this so wrong – and why a bit of common sense, as well as some familiarity with HSR around the country and around the world, would make all the difference.
We’ll start at the top – in Sacramento, where the Bee’s Dan Walters, a longtime opponent of high speed rail, clearly gets it wrong in his column on the study:
If the projection is unrealistic, the bullet train could become an expensive sinkhole.
From its inception, the project has appeared to be a political boondoggle – a solution in search of a problem. The UC report is the latest bit of evidence to that effect. Unless the gaping holes in its viability can be bridged, the bullet train should be derailed.
As you can see here, Walters has no concept of things like high gas prices, traffic, or thegreen dividend and job creation HSR will provide. Walters is one of the classic examples of someone who is stuck in the 20th century, who cannot and will not believe his own eyes, which should tell him that the car and the plane will not always be the only way people get around California. Ironically enough, Walters is very forward-thinking on reforming state government, showing a willingness to be quite creative and open to new ideas about how to fix Sacramento. But when it comes to transportation, his blind spot is large.
At the San Francisco Chronicle, where their print article does a good job on the story (more on that below), their Bay Area Transit blogger, Nathanael Johnson, shows a stunning lack of knowledge of HSR that, in my mind, should cause the Chronicle to question whether Johnson is qualified to write on transit-related topics:
But if you wade into the report, which can be found here, it’s clear that much of the professors’ critique has to do with assumptions made by Cambridge Systematics (CS) which seem to defy real-world experience.
Only if your “real-world experience” doesn’t include HSR. An example:
Here’s a good example of the problems they are talking about – and this gets a little complicated, but it’s interesting: CS assumes that people will show up at train stations and wait for the next train to arrive. If that’s what people do, then the time between trains, or headway, will really impact its usefulness. But when I make a long distance trip I plan ahead, check the schedules, and arrive a little before the train or plane departs.
Johnson makes the same error Samer Madanat makes, which is to assume people’s behavior with a bullet train will be the same as a plane. It won’t. In the comments to yesterday’s post, Matthew wrote that people actually do just show up at an HSR station and wait for the next train, at least in his experience in Germany:
Having the schedule reasonably redundant made up for this pretty well, and I was rarely delayed by an hour or so, and usually not delayed at all. Pleasant cafes made the wait not such a problem. I usually would coordinate with the schedule, but occasionally would just show up at the station, especially if I was just going to a nearby destination.
I’ve heard of this happening on the Acela as well – people going to Union Station in Washington DC and just grabbing the next train out. It is an entirely plausible model –because it happens around the world. Madanat has a history of ignoring real-world HSR examples in favor of his own theoretical views, even when his theories clash with real-world realities.
Johnson makes a much bigger error – one that I believe should cause us to question whether he is suited to write on transit issues at all – when he questions the HSR ridership numbers in comparison to the Acela:
Planners assumed that trains would travel less frequently on Altamont option, which would increase wait times, and decrease ridership. A lot.
“The sensitivity to train frequencies penalized the Altamont routing by 20 million riders per year. The entire ridership of the Northeast Corridor Amtrak service is approximately 10 million riders. The report suggests that the sensitivity may have been over-inflated by 4 to 5 times.”
The northeast corridor, which runs from Washington D.C. through New York to Boston, is the busiest passenger rail line in the United States. Is it really reasonable to expect that having trains run twice as often would produce 20 million new riders?
Yes, it is reasonable to expect that, and if you do not believe that is reasonable, then you have no business writing about transit for a major newspaper, even as a blogger.
Johnson and Madanat trade on the notion that “the Acela only does 10 million, so obviously anything higher than that is just not credible.” That is quite simply a bullshit argument. The Acela is actually a limited form of HSR that does not achieve the speeds or the capacity of California HSR. HSR systems around the world routinely carry more than 10 million riders. California’s will too, assuming we build trains that can cover the SF-LA route in under 3 hours and connect city centers to city centers with frequent service.
Any argument that says “the Acela is the best the US can do and any projection higher than that is not worth taking seriously” is a dishonest argument that flies in the face of logic and the evidence.
Johnson also writes about a quote Mike Rosenberg got from Alan Lowenthal in his own article in the Mercury News, which is overall a good article with an unfortunate headline. We’ll deal with Lowenthal tomorrow; he deserves a post all to himself.
Another article comes from a serial offender – Tracy Wood of the Voice of OC. In an article illustrated with a picture of empty train seats (get it? because supposedly this report means nobody will ride the trains?), she buys hook, line and sinker the criticisms made of the HSR project:
But under the state law that voters approved in 2008 that authorized construction of the system, it must pay for itself once it is finished.
To do that, according to the ridership study released earlier this year, the high-speed system would actually have to rely on a large number of local commuters and compete with existing commuter rail systems for fares.
Its business plan estimates almost one-third of the expected 41 million annual riders will stay within the Los Angeles basin or in the San Francisco Bay area.
Wood writes this with an attitude that indicates “of course this expectation is absurd.” But is it?
Within the SF Bay Area, it would seem logical to expect a lot people turning to HSR to complete a journey from SF to San José – it would be significantly faster than even the fastest Caltrain service. Similarly, anyone looking to go from central Orange County to central LA (or to areas connected to central LA by Metro Rail) would likely take HSR, same with people going from Riverside/San Bernardino to central LA. It’s a plausible concept, and if Wood wants to bash it, she should provide evidence, instead of just assuming we all share her view that nobody rides trains. (Has she been on a Metrolink or Pacific Surfliner train in OC before?)
Not surprisingly, having some experience with HSR leads to a more balanced and accurate report. Michael Cabanatuan of the SF Chronicle offered this fair assessment in today’s paper, making it clear that this dispute is not about whether the books were cooked:
Madanat said, however, that the report’s conclusions and criticisms should not be seen as evidence Cambridge Systematics rigged its report to show higher ridership figures, as some high-speed rail opponents have suggested.
“This is the best firm in the business,” he said. “They have a reputation to protect. I would not say, and I would have a hard time believing, that they skewed the numbers. And there is no evidence of that.”
And as I mentioned, Mike Rosenberg did a good job too.
One common feature in all these articles: Senator Alan Lowenthal. His quotes are, quite simply, shockingly inappropriate. As I said, he deserves his own post, and he’ll get it tomorrow.
