TalkTuahBank

Multi-Agent, Conversational AI, & Telephony Integration

<p align="center"> Frontend built with: <br> <img src=https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB alt="React"> <img src=https://img.shields.io/badge/TypeScript-007ACC?style=for-the-badge&logo=typescript&logoColor=white alt="Typescript"> <img src=https://img.shields.io/badge/next%20js-000000?style=for-the-badge&logo=nextdotjs&logoColor=white alt="Next"> <img src=https://img.shields.io/badge/Tailwind_CSS-38B2AC?style=for-the-badge&logo=tailwind-css&logoColor=white alt="Tailwind"> <img src=https://img.shields.io/badge/shadcn-000000?style=for-the-badge&logo=shadcnui&logoColor=white alt="Shadcn"> <br><br> Backend built with: <br> <img src=https://img.shields.io/badge/JavaScript-323330?style=for-the-badge&logo=javascript&logoColor=F7DF1E alt="Javascript"> <img src=https://img.shields.io/badge/Node%20js-339933?style=for-the-badge&logo=nodedotjs&logoColor=white alt="Node.js"> <img src="https://img.shields.io/badge/OpenAI_Swarm-34a853?style=for-the-badge&logo=openai&logoColor=white" alt="OpenAI Swarm"> <img src="https://img.shields.io/badge/Pinata_Cloud-F2E3A1?style=for-the-badge&logo=pinata&logoColor=black" alt="Pinata Cloud"> <img src="https://img.shields.io/badge/Retell_AI-FF7F50?style=for-the-badge&logo=retell&logoColor=white" alt="Retell AI"> <br> </p>

🏅 HackUTD - Overall 1st Place & 1st Place Sponsor Challenge: Goldman Sachs

<br> <p align="center"> <a href="https://devpost.com/software/talktuahbank?ref_content=user-portfolio&ref_feature=in_progress"> <img width="1728" alt="Talk" src="https://github.com/user-attachments/assets/46459527-c8c5-4379-a0b4-e6aac1b96d76"> </a> </p>

Inspiration

Access to banking services is a fundamental aspect of economic participation, yet over 1.7 billion adults worldwide remain unbanked. Traditional banking solutions often require internet access, digital literacy, or proximity to physical branches—barriers that leave underserved populations without essential financial tools. Inspired by the need for inclusivity and the power of conversational AI, we envisioned TalkTuahBank, a voice-based banking assistant that democratizes financial services by making them accessible to everyone, regardless of their technological capabilities or geographical location.

What It Does

TalkTuahBank revolutionizes the banking experience by offering a fully conversational AI assistant accessible through simple phone calls. Users can effortlessly manage their finances by performing tasks such as checking account balances, transferring funds, and paying bills using natural voice commands. Key features include:

• Voice-Activated Services: Conduct banking transactions hands-free via phone.
• Multi-Language Support: Interact in multiple languages and dialects to accommodate diverse users.
• Accessibility: Operates on any phone without the need for internet or smartphones.
• AI Assistance: Receive personalized financial advice and support through intelligent interactions.

How We Built

We developed TalkTuahBank by integrating cutting-edge technologies to create a seamless and secure telephonic banking experience:

• Conversational AI: Utilized Retell AI for natural language understanding and generation, enabling smooth voice interactions.
• Multi-Agent Framework: Implemented OpenAI Swarm for agent orchestration to manage complex dialogues and backend processes.
• Secure Data Storage: Leveraged Pinata (IPFS) for decentralized and secure storage of user data and transaction records.
• Admin Dashboard: Built with Next.js, Tailwind CSS, ShadCN UI, and Aeternity to provide a robust interface for system management, transaction monitoring, and user support.
• Telephony Integration: Employed Retell API for handling voice calls and processing voice recognition.

Challenges We Ran Into

Developing a voice-based banking system presented several hurdles:

• Security Concerns: Implementing robust security measures to protect sensitive financial data was paramount, necessitating the integration of advanced authentication methods and decentralized storage solutions.
• Integration Complexity: Seamlessly connecting multiple technologies (Retell AI, OpenAI Swarm, Pinata, etc.) into a cohesive system demanded meticulous planning and execution.

Accomplishments That We're Proud Of

• Functional Prototype: Successfully developed a working prototype that allows users to perform essential banking tasks through voice commands.
• Multi-Language Support: Enabled interactions in multiple languages, enhancing accessibility for a diverse user base.
• Decentralized Data Storage: Implemented Pinata for secure and scalable data management, ensuring user information is protected and easily retrievable.
• Comprehensive Admin Dashboard: Created a user-friendly admin interface for managing users, monitoring transactions, and providing support, built with modern frameworks like Next.js and Tailwind CSS.

What We Learned

Throughout the development of TalkTuahBank, we gained valuable insights:

• Voice Technology Nuances: Deepened our understanding of voice recognition technologies and the challenges of creating natural, intuitive voice interactions.
• Security Best Practices: Learned the importance of multi-layered security approaches to protect sensitive financial data effectively.
• Integration Strategies: Developed expertise in integrating diverse technologies into a unified system, ensuring seamless functionality and performance.
• User-Centric Design: Recognized the critical role of designing for accessibility and inclusivity to meet the needs of a varied user base.

What's Next for TalkTuahBank

Looking ahead, we aim to expand and enhance TalkTuahBank to further empower underserved populations:

• Expanded Language Support: Add more languages and dialects to cater to an even broader audience.
• Fraud Detection: Implement machine learning-based algorithms to identify and prevent fraudulent activities proactively.
• Financial Education: Introduce educational modules to help users improve their financial literacy and make informed decisions.
• Mobile App Integration: Develop a complementary mobile application for users who gain access to smartphones, providing a seamless transition between phone and app-based banking.
• Partnerships and Compliance: Strengthen partnerships with local banks and NGOs to expand our reach, while continuously ensuring compliance with evolving financial regulations.
• Enhanced Security Measures: Continuously upgrade our security protocols to stay ahead of emerging threats and protect user data comprehensively.