Generative AI

Started in August 2023

AI voice cloning is already being used for spear phishing. Soon, companies may be targeted on a large scale by criminals. Therefore, it is of crucial importance that PCSI partners are adequately prepared for AI voice calls. 

https://pcsi.nl/uploads/projects/generative_ai_1.png

Project Proposal 

The aim of the project is to develop a solution that provides the call recipient (e.g. helpdesk, customer service, engineers, procurement officers) with an indication that the caller may be using AI to generate or disguise a voice. 

Expected benefits of the Generative AI project? 

Besides the voice detector itself, by collaborating with PCSI partners, we can identify the main requirements needed to develop such technology, and the way different organizations can be affected by AI voice calls. 

Moreover, the team and the partners will acquire knowledge that will enable them to keep up with the latest developments in the field.  

Why do we want to work on this idea within the PCSI? 

Build a shared knowledge base on deepfake audio detection. Develop a solution with input from multiple partners. The project is open to liaison partners and follow-up proposals. 

Our use-case: Helpdesk calls 

Whether used as targeted spear-phishing technique, or to "DDoS” a helpdesk (keep all operators busy), helpdesks will surely be targeted by AI generated voices. We want to develop a tool that can help the receiver determine in real time if the caller is an actual person or a generated voice. 

Activities in Explore phase 

  • Discover what the state of the art is for vishing attacks 

  • Investigate latest commercial and open-source tools already available to create AI voices 

  • Obtain training datasets 

  • Plan for a solution implementation 

Activities in Proof of Concept phase 

  • Identify features useful to distinguish real from fake audio 

  • Implement first prototype of model 

  • Train and test model on training datasets 

Activities in Pilot phase 

  • Validate performances of existing features 

  • Keep developing model 

  • Test model at partner's side, with partner's generated data 

Activities in Exploit phase 

  • Refine model based upon insights from ASVSpoof5 challenge 

  • Finalize integration at partner's site 

Beeldmerk PCSI
PCSI is een samenwerking van
    ABN-AMRO Achmea ASML Belastingdienst ING TNO