14 Jan 2010
Web scraping changed the Travel and Transport Industries
The battlefield
The heavy pace of recent financial downturn highlighted Travel Industry requirements to focus on business models.
In traditional air transport, just to give a few examples, the continuous need to streamline operations due to structural losses; the necessity to negotiate new employment agreements; the inability to implement savings without substantial manpower cuts; signs of a demand recovery where ignition is almost unpredictable.
More in general, the need to reconsider the structure of a business that – albeit is assisted by a hope of recovery – cannot but take into account the most crucial recent experience: the advent of Low Cost Carriers, together with the subsequent reduction of the price charged by the Airline in the near application period, have accustomed the final users to lower and lower fares, constantly sought over the Internet.
A scheme that has also helped the proliferation of new players among the Tourism Industry (i.e. Web-Based Tour Operator, aka OLTA). It is therefore clear, in this scenario, the strategic importance of pricing policies directed to a more attentive and informed Customer Base, for which even a small difference between two bids lead the choice of purchase.
Internet: the battlefield changed, definitively.
The challenge
Pricing policies?
Oops… we are back to the old dear price!
But, aren’t we talking about business models? Where is the link?
Well, there’s no business if we can’t generate profit, there’s no profit if we can’t sell goods and/or services and… price is that part of our proposal which Customers are paying more attention to in a penny-pinching era!
Price: on bills, magazines, shelves, shop windows or web pages, it is determined by a number of factors ranging from production cost to market conditions.
In line with corporate profit maximization theories, it is necessary to focus on understanding pricing policies implemented by The Net, analyzing the ensuing economic – and social – impact.
Result? In the 21st century, Travel Industry has to daily face the gap between Profitability and Customers Loyalty, often hedging within few pennies.
Perpetually, without pause!
Practically speaking, Airlines need to implement algorithms, procedures and hi-tech tools to really be able to dynamically support new business policies.
We are not only speaking about revenue and yield management choices: there’s something new in The Game, a sort of market intelligence evolution.
It’s a must: Airlines cannot but rethink price modelling because their choices need to be supported by constant monitoring of competitors.
Comparison is required for the appropriate adjustment to their own offer.
Comparison is made obligatory to evaluate real-time changes in market price.
Real-time comparison is needed to act, for example, with the aim of minimizing the gap between our offer and the collected fares.
That’s a brand new, real source of revenue.
The rule book
In the recent past, some Airlines and Hotel chains felt the need to implement price policies based on comprehensive and updated competitors’ data.
This has led many companies to seek solutions through working groups dedicated to the so-called “harvesting of prices”.
Others chose to develop more or less sophisticated software components for automatic collection.
Some firms have attempted to run away from competitors automated detection by structuring their sites so that the traditional collection systems are automatically rejected and actions from suspicious Internet addresses are made harmless.
Others still monitoring simply the traffic on their sites with the aim of understanding WHO is watching WHAT, without implementing defence policies.
The emerging data is that security policies are being refined and – with regards to the effect of direct and indirect sales – the market acquirement has already moved on technological field, where competitive advantage is the possession of the most innovative, technical and cultural know how.
The challenge is played at various levels and requires a holistic approach.
Here is some advice for you from our point of view.
a) Collect and Process
According to a pure price marketing logic, competitors’ current price availability means that we can offer to the Customer Base the lower fare on the market and, therefore, close the sale positively.
In practice this is not always systematically applied, nor is it recommended. In fact, sales techniques require most comprehensive support algorithms to deal with market demand, to handle seat availability, to assess change in revenues per unit on the basis of the real trend in demand: in other words, Revenue and Yield Management along with real-time pricing.
Nevertheless, real-time availability of competitors’ web fares may definitely widen the range of data required for a more careful planning of actions aimed at implementing the pricing strategy: through this it becomes possible, for example, the minimization of the gap with the major competitor and/or the publication of a price slightly higher than those of competitors.
Whatever the action plan after enforcement may be, what mostly counts it’s to speed up Internet data collection process (web data retrieval), which is essential to accelerate subsequent elaboration in the mix of information owned and collected: this is the only way we can be sure to act in a timely fashion with the purpose to modify our offer in response to changes made by competitors.
b) Dissimulate
The act of retrieving data published on competitors’ websites is usually made explicit by a number of instances – geared towards the target system – of such amplitude and intensity to endanger the identity of the applicant.
The risk is to be identified by a competitor’s control system that might be more than troublesome when/if the competitor decides to convert its defence into scam actions: the results could be devastating!
The automatisms taken to avoid detection are based on techniques of concealment that allow the evasion from control systems through the proposition of identities and/or characteristic behaviours not attributable to the applicant.
c) Identify Competitors
As it is essential to monitor competitors, you should be ready to avoid similar behaviour on their part.
The first step in this direction is to identify the enemy.
The recognition is a process-based mechanism, whose analysis and implementations go along with a high level of know-how. The investigation (and the subsequent action plan) may be simple or very complicated: it depends on the level of complexity of the methods used for the unmasking and the quantity and quality of data available for the scrutiny of the entities’ behaviour who surf the web pages that arouse competitors and Customer Base attention.
d) Block and/or Scam
Competitors are not just watching.
On the web, the battlefield is extremely tricky.
You should not only consider actions to protect the chosen strategy and adopted tactics, but evaluate the overall effects of the data collection operations made by competitors: they can lead to an actual interruption of your service, occupying the entire available bandwidth and slowing down the traffic as far as the time when the web surfing of the Customer Base becomes impossible. A problem that, if encountered during high season, holidays, promotions, etc., undergoes multiplication factors with the risk of producing results even more deleterious (e.g. competitors can draw a greater – albeit illegal – profit!).
In this competition data retrieval is one of the most effective plans of attack: the defensive aspect is made explicit (in its most extreme) in a set of opportunities that can even provide real trick manoeuvres.
Identifying competitors, unequivocally, is the first step: here you can choose to stop the flow of requests from competitors or vehicles to sites that provide erroneous data. Moreover, the responses consciously designed to address competitors are – in turn – a technique used to assess their reactions.
Concentration, Relocation and Security
Websites and related sales systems are 24/7 accessible.
Similarly, and also to gain further competitive advantage in times of lower labour presence, a price marketing system should work seamlessly.
This requires a dedicated space which is – in addition to being a convenient logistic container – the integration centre of activities in order to provide an immediate visibility, facilitating internal communication and supporting management decision making.
To this end, and also for the use of systems during travels and/or closings, it’s necessary to provide discriminate and secure access, planning (but, when necessary, overcoming) the restriction to external authorized workstations.
Automating response processes and their oversight
Enhancing potential use of systems means – where inapplicable a 24/7 continuity – mechanizing the price loading/issuing.
Such changes should be automatically managed and certified, or – when and if necessary – supervised and guaranteed by manual notification from those operating officers entitled to intervention: that can be conveyed by the more easily accessible device (e.g. PDAs or smartphones, mobile phones and SMS messages, notebooks and e-mail or browsing, etc.).
So, is your Yield & Revenue Management strategy working for you?
Did you ever consider to take a look at how web scraping impacted on consumer fares?
We would like to know more about your experience and have your opinion.
Gianni Tolu
email gianni.tolu@axoluta.com
web Axoluta® Website
LinkedIn Gianni Tolu
Skype gianni.tolu

