Definitely Doug 10/18/24: Facial Recognition and Authentication: Is Now the Time?

‍

Biometrics technologies in general, and facial recognition and authentication in particular, are complex and rapidly improving. They can reduce many risks for hotels and improve personalization of service, but they can also create data security risks and breach liabilities if not properly managed. This is not a technology that I would recommend universally today, but it is starting to make sense for more and more hotels as it gets better and cheaper. Today’s column will try to parse the issues so you can evaluate whether the time is right for your hotel to take a closer look.

Facial recognition and authentication are related but different technologies. Facial recognition typically uses stored biometric data against a live camera feed to identify individuals who appear on the camera and whose biometric facial profiles (digital representations of their face) are stored in a database. Facial authentication is used to verify someone’s identity, typically by comparing the biometrics of a face in a live photo or video stream to one depicted on an identity document such as passport or driver’s license, or in a database.

These technologies are becoming more appropriate for many hotels than in the past, depending on your market segment, location, and other factors. Key changes include the following (assuming guest consent where required; more about that later):

• A significant reduction in cost, as software matures and increasingly reuses already-in-place cameras and mobile devices, and commercially available AI video processing software

• Ability to prove that a credit card was presented by the actual cardholder, reducing chargeback risk

• Better functionality from products designed to identify a guest approaching a staff member, to display the guest’s photo to help the staff recognize them, and to display context-relevant personalization information (such as food allergies at a restaurant or beverage preferences at the bar)

•  Increased deployment of digital wallets (planned for release and government support in all EU countries by 2026), enabling guests to control the sharing of their biometrics (such as passport photo) digitally rather than by providing the physical document at check-in and requiring the hotel to retain a coy

•  A reduction or elimination of the need for hotels to store sensitive information (such as those passport photos), as well as far better security where the need remains

• Better physical security within the hotel from the ability of the cameras and video analytics to recognize anomalies that may pose threats

• Growing support from providers of hotel property management and mobile app software

• Better identification of repeat guests who may book through different distribution channels, making them frequently difficult to deduplicate in customer databases

‍

Beyond identification and authentication, there are other benefits from deploying the cameras and software that support facial biometrics. Some hotels are using these capabilities to measure crowd sizes in common spaces, to count the number of people in a queue, to detect the presence of room service trays in the guest-room corridors, to recognize guest license plates as they approach the porte cochère, or to report medical issues such as a guest or staff member who has fallen in a monitored area.

I am indebted to leaders at several companies for the research behind this article, including Youverse, which provides privacy-preserving face authentication solutions used by both banks and hotels, and Neoma, which uses facial recognition to enable more personalized guest service, especially in luxury hotels. In past discussions regarding facial authentication for mobile and kiosk check-in, several other companies provided me with insights that also informed today’s article, including Canary Technologies, Duve, hudini, and Virdee.

Privacy and Consent Considerations

Guests often have strong opinions about the use of their biometrics, and there are a few rules for using it without violating their privacy and trust. First and foremost, hotels should always get consent before obtaining or storing any guest biometric information that is not required by the authorities.

However, expectations of privacy (as well as regulations) vary significantly in different parts of the world. Residents in some countries know that they are under constant video monitoring with facial recognition in public, so they may view any additional risk as minimal. Others may not want to share biometrics even in return for significant tangible benefits. Most people fall somewhere in between: they are willing to share information if they believe it will make their hotel experience more personalized or enjoyable, and that it will not be abused. Their willingness to share with a specific hotel may also depend on how and when they are asked, and how much they trust the hotel or brand.

Storing biometric information, beyond what is sometimes necessary to meet government requirements, can be risky in the event of a data breach. However, new technologies (sucy as from Youverse) enable biometric profiles to be stored in a way that renders them useless unless they are combined with other data that is stored securely on the individual’s mobile device. Similar to the way in which tokenization made it safe for hotels to store payment card data, the latest approach makes it safe to store biometrics. The traveler and the hotel each have information from which a biometric profile can be created, but (a) the hotel cannot do it without the portion held by the traveler; (b) the traveler controls whether to share this with the hotel; and (c) the hotel gets assurance that the traveler’s portion was authorized by them and not by someone else.

There are also many use cases around physical security where consent may not be required, or where it can be obtained as part of a broader relationship (such as employment). For example, hotels can store employee biometrics and use it to control access to back-of-house areas or to staff devices such as computers, point-of-sale terminals, or staff telephones.

Use Cases for Facial Recognition and Authentication

There are several common use cases where facial recognition or authentication are already used in hospitality (even widely, in certain countries or regions), and others that are emerging. These include:

Mobile and kiosk check-in: In jurisdictions where identity verification is required at check-in and data from (or images of) documents sent to local authorities, more and more are starting to accept fully digital processes (as opposed to requiring the hotel to make paper or digital copies of physical documents). Typically, the identity document is scanned (or tapped, if chip-enabled), and a camera captures the facial image and compares it to the one displayed on the document or encoded on the chip to ensure a match. For mobile check-in on a phone, there can be additional checks to ensure that the user is providing a live image rather than holding a photo in front of the camera. Biometrics supported by the phone, such as FaceID, may provide further verification. Scanned or digital identity documents can in many cases also be checked for tampering and/or validated by the issuing agency in real time. With the guest’s consent, the photo can also be captured for the hotel to use to help identify the guest throughout their stay.

Front desk: For luxury and higher-end hotels that want to deliver highly personalized service, the front desk is another opportunity to capture the guest photo to support staff recognition. Check-in (whether mobile, kiosk, or front desk) is the most logical place to get the guest’s consent to use their photo for this purpose. The CCTV camera behind most front desks can (through integrated software) pop up a window on the front desk terminal where staff can select (e.g. via touch) the location of the guest’s face to capture their photo and store the biometric profile.

Loyalty account onboarding and security: Loyalty accounts are essentially bank accounts that store a private currency. Large accounts are routinely targeted by cyber criminals because the currency has value. While most loyalty transactions are low risk, some (such as the withdrawal of millions of points) are not; biometrics can greatly reduce the risk for high-value transactions. This requires an onboarding process (either when the account is first opened or at least before the point balance gets too large) where the biometric profile can be collected and verified by comparison to an identity document. The stored biometrics (preferably using the security approaches described earlier that make them unusable until combined with information stored on the member’s mobile device) can then be used to ensure that only the account holder can access the account. If the account holder loses access (for example because they lose their phone), repeating the onboarding process (with identity document verification) can restore it with minimal risk to the program operator.

Guest recognition and personalization: Luxury hotels have for many years posted photos of VIP guests in the back office for staff to scan. This enables them to recognize guests and personalize their stay. But there is a limit to how many photos most staff can memorize. With facial recognition, the process can now be automated for all guests who have consented.

If a guest approaches hotel staff at the front desk, restaurant hostess station, bar, executive lounge, concierge, bell desk, health facilities, or elsewhere, facial recognition can be used along with the feed from a closed-circuit television (CCTV) or other camera to identify them and to populate relevant contextual information on their screen. This might include seating or food preferences or allergies at a restaurant, activities of interest at the concierge desk, or a recap of a recent service incident that might warrant special recognition or acknowledgement by staff.

While I have not seen it yet, I expect that Artificial Intelligence (AI) will soon come into play to help curate all the information about the guest into the few tidbits that are most relevant to hotel staff in a given context.

Monitoring common spaces: The same technology that can recognize faces can also work anonymously to identify crowd situations or queues that may require staff attention, or to identify guests who are loitering or exhibiting odd or concerning behavior or medical emergencies, and alert hotel security.

Back of house security: If facial biometrics are obtained for employees and others who should be whitelisted for back-of-house access, it can be used to authenticate and authorize access to locked areas (typically using a camera mounted in an access control device on the door frame). Alternatively, CCTV cameras can passively identify individuals entering back-of-house areas and raise alerts to security staff if any non-whitelisted people enter the area.

It is also possible to use biometrics to secure staff devices such as point-of-sale terminals, telephones, and mobile devices, using an embedded or nearby camera. This can, for example, prevent someone from using a stolen point-of-sale swipe card from accessing the till. It can also secure public-area staff phones that may be left unattended at times from being used by non-staff. There are known and serious cases of stalkers using such phones to call the front desk and ask the room number of a guest; because the call displayed as an “internal” call, they were able to get their target’s room number that should not be given to non-staff.

Related use cases: For some applications, facial recognition is just one option for identifying guests and not necessarily the best one. For example, wristbands using Bluetooth or near-field communication (NFC) technologies are often issued to guests on cruise ships or in resorts, which use them to identify guests as they approach staff members. For example, a bartender’s screen might display pictures of every guest with such a device in range, to enable personalization similarly to using facial recognition.

Technology Considerations

Biometric data is personally identifiable information (PII) and subject to stringent regulation in many jurisdictions. While it is possible to store it securely, hotels are not known for their ability to secure data well, especially at the individual unit level. The better approach uses cryptography to encode and split the data into two parts, which must be combined to be usable. The hotel holds one part, the guest holds the other. Neither piece alone constitutes PII or is useful without the other, so simpler security measures are sufficient.

Even the most secure approaches to using facial recognition for authentication are hackable, however. An arms race has developed between facial recognition applications and bad actors who use AI video generation to overcome challenges like liveness tests. For example, a liveness test might ask a person to point their phone camera at their face and then respond to random spoken instructions, such as to smile or wink. AI, however, is rapidly reaching the point where it can generate real-time video that responds just like a human, and that video can be displayed on another device that is held in front of the phone’s camera. The better technologies can currently detect most such crude attempts and foil them, but it can be harder to detect AI fakes if the criminal hacks into the camera network and sends the video feed digitally.

Security vs. Cost Tradeoffs

You cannot get perfect security; a determined and capable criminal with a large enough target can ultimately prevail, although it will be far more difficult than the old-school method of passing off a fake identity card at the front desk. Hotels should try to match the level of security to the risk. If the risk is low, “fairly secure” may suffice; where the risk is higher, you can up your game as much as you like.

In terms of cost, you can spend more or less on the facial recognition technology itself, on connected CCTV systems (which are typically used for other purposes as well), and for identity document verification. The latter is often a variable cost; simple format validation, tamper detection, and information capture may be in the pennies per check, while full real-time verification of authenticity with the issuing agency may cost several dollars. The latter is probably overkill for most hotel use cases, although it might make sense for employee onboarding, for loyalty accounts that exceed a certain value, or for casino high-rollers. For hotel check-ins, high security is not usually needed or cost justified; you typically just need enough proof that the cardholder was present at the time of the transaction to be able to fight chargebacks effectively. But there could be exceptions, for example when renting a high-value suite with expensive furnishings or artwork.

The emergence of digital self-sovereign wallets and identity documents will soon start to replace the need for real-time verification of paper documents, although the timeline appears likely to vary widely in different parts of the world. Especially for hotels in the European Union and other locations where governments are implementing these, real-time physical document verification solutions may be short-lived. The facial authentication processes do not necessarily change with this transition; rather, the source of the identity document photo to be matched changes from a scanned physical document to a digital one (probably presented securely via NFC). It is worth understanding how any potential vendor is prepared to address this transition.

Conclusion

Most hotels today that offer mobile or kiosk check-in can potentially benefit from lightweight applications of facial authentication and document verification to verify the identity of the guest, both as a good business practice and to minimize chargeback risk. Luxury hotels can do much more to facilitate guest recognition and personalization, as many of the above use cases illustrate.

There is no one-size-fits-all solution, and the technology costs will still likely come down in coming years; depending on your use cases an adequate return on investment may exist today or might need more time. But this is relatively simple technology that reuses devices and infrastructure many hotels already have in place, meaning it can often be deployed at modest incremental costs.

As always, feedback to my articles is welcome. Since the host site does not support discussions, I will post a link to this article on my own LinkedIn page once it has been published, and I invite you to comment, like, or share from there!

‍