Don’t you think that a great many mobile apps would be a lot more convenient if they had voice control? And I don’t mean chatting with a banking bot. In most cases, voice navigation or a conversational form-filling is just enough. Through the use of the example (an open-source Kotlin-based habit tracking app) Vit Gorbachyov, Just AI solution architect, will show you how to add a voice interface into any app swiftly and seamlessly. Habitica How convenient does that sound? Let’s start with the obvious: It’s really often that we need to use apps when our hands are full – when we cook, drive, carry a suitcase, etc. Voice is a major instrument for people with vision disabilities. It’s really obvious, but most of the time voice is simply quicker. Consider this, ordering a ticket saying instead of a long-time form filling. With an option of clarification questions – should that be morning or evening? Will there be luggage or not? get me a plane to London for tomorrow for two Voice is very useful in a form-filling scenario and it suits perfectly almost any long forms, requiring lots of info from a user. And these are the kind of forms that almost any mobile app has. Most companies keep trying to stuff all the functionality they can think of into a chat because usually voice assistants are embedded into chat support. You know, to refill the balance, get the item or service info, etc. It’s not always convenient, sometimes it’s even counterproductive because voice recognition is yet not perfect. The right approach here is to embed an assistant into the already existing app functionality. So we took as an example – it’s perfect for a voice assistant addition because in order to create a new task you have to fill in the long form. Let’s try to swap out this dreary process to one phrase with some guiding questions. Habitica What we need to get started . We use to create dialog interfaces. It’s an open-source voice assistant SDK with ready to use . You can use built-in and components and or you can create your own suite. SDK Aimybox customizable UI speech to text text to speech NLU implementations Aimybox implements the assistant’s architecture, standardizes interfaces, and organizes mutual support. So, you can really cut down the time it takes to develop a voice interface. . We will use open-source Kotlin-based chatbot and voice assistant development framework (Just AI Conversational Framework), which is totally free. (an NLU service) inside the (Just AI Conversational Platform) would help us with intent recognition. Tools to create a scenario JAICF Caila JAICP I’m covering these tools in the . There I show how to manage an application with voice only –invoking certain screens, implementing complex queries within the app, and changing habits. next part of this tutorial . We will need an Android phone to test the Habitica solution. Smartphone Action plan We start with a project fork – we take the Release development branch and look for the most essential files. I used the IDE Android Studio: .kt – that’s where we build the logic in. MainActivity .kt – that’s where we will initiate Aimybox HabiticaBaseApplication .xml — that’s where the interface element will be   AndroidManifest.xml — that’s where the app’s structure and its permissions are stored Activity_main According to Habitica’s repository instruction, we rename and by taking the out. Then we start a new firebase project and copy the file to the root node. habitica.properties.example habitica.resources.example example google-services.json Now we start the app to see whether it works. Ta-dah! To begin with, we add Aimybox dependencies. implementation implementation( ) 'com.justai.aimybox:core:0.11.0' "com.justai.aimybox:components:0.1.8" to dependencies maven { url }
  maven { url } 'https://dl.bintray.com/aimybox/aimybox-android-sdk/' "https://dl.bintray.com/aimybox/aimybox-android-assistant/" and repositories. And right after compileOptions we add the following lone so that everything works properly. kotlinOptions {
        jvmTarget = JavaVersion.VERSION_1_8.toString()
    } Now the permissions. We take the flags off in and in .xml, so that the options look like that RECORD_AUDIO MODIFY_AUDIO_SETTINGS AndroidManifest <uses-permission android:name= /> <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <uses-permission android:name="com.android.vending.BILLING" />
    <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>
    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/> "android.permission.READ_PHONE_STATE" < = /> uses-permission android:name "android.permission.ACCESS_NETWORK_STATE" Now we initiate Aimybox in a BaseApplication.  Adding AimyboxProvider at the component initialization And initializing: private fun createAimybox (context: Context): Aimybox {
        val unitId = UUID.randomUUID().toString()
        val textToSpeech = GooglePlatformTextToSpeech(context, Locale( ))
        val speechToText = GooglePlatformSpeechToText(context, Locale( ))
        val dialogApi = AimyboxDialogApi( , unitId) Aimybox(Config.create(speechToText, textToSpeech, dialogApi))
    } "Ru" "Ru" "YOUR KEY" return Afterward, instead of YOUR_KEY, you will see your Aimybox Console code. Now we build in a fragment into mainActivity.kt. We embed FrameLayout in activity_main.xml tentatively, right under the FrameLayout with id bottom_navigation <FrameLayout
             android:id= android:layout_width= android:layout_height= /> "@+id/assistant_container" "match_parent" "match_parent" Into the MainActivity we add OnCreate explicit request permissions ActivityCompat.requestPermissions( , arrayOf(android.Manifest.permission.RECORD_AUDIO), ) this 1 And when we get it, we add the fragment into the frame mentioned above. @SuppressLint( )
    override fun onRequestPermissionsResult(
            requestCode: Int, : <out >, : IntArray
    ) {
        val fragmentManager = supportFragmentManager
        val fragmentTransaction = fragmentManager.beginTransaction()
        fragmentTransaction.add(R.id.assistant_container, AimyboxAssistantFragment())
        fragmentTransaction.commit()
    } "MissingPermission" permissions Array String grantResults Don’t forget to add an option to log out the assistant after you’ve logged in into OnBackPressed. val assistantFragment = (supportFragmentManager.findFragmentById(R.id.assistant_container) ? AimyboxAssistantFragment) (assistantFragment?.onBackPressed() != ) { } as if true return Besides, we add styles to styles.xml in AppTheme <item name= >@style/CustomAssistantButtonTheme< CustomRecognitionWidgetTheme< CustomResponseWidgetTheme< CustomImageReplyWidgetTheme< CustomButtonReplyWidgetTheme< "aimybox_assistantButtonTheme" /item>
        <item name="aimybox_recognitionTheme">@style/ /item>
        <item name="aimybox_responseTheme">@style/ /item>
        <item name="aimybox_imageReplyTheme">@style/ /item>
        <item name="aimybox_buttonReplyTheme">@style/ /item> And some custom styles a bit underneath: <style name= parent= > <style name= parent= > <style name= parent= > <style name= parent= > <style name= parent= > "CustomAssistantButtonTheme" "DefaultAssistantTheme.AssistantButton" </ > style "CustomRecognitionWidgetTheme" "DefaultAssistantTheme.Widget.Recognition" </ > style "CustomResponseWidgetTheme" "DefaultAssistantTheme.Widget.Response" </ > style "CustomButtonReplyWidgetTheme" "DefaultAssistantTheme.Widget.ButtonReply" </ > style "CustomImageReplyWidgetTheme" "DefaultAssistantTheme.Widget.ImageReply" </ > style Now let’s see whether the mic has appeared. Launching an application. Okay, we got plenty of syntactic errors. We correct everything as IDE says. Aaaaand it works! But the mic slipped down the navigation. Let’s give it a lift. Let’s add this to the styles in CustomAssistantButtonTheme: <item name= > dp< "aimybox_buttonMarginBottom" 72 /item> That’s better! Now we switch in the assistant to see whether it responds well. We will need the Aimybox console for that. We go to app.aimybox.com using Github acc, create a new project, add a couple of skills (I’ve added DateTime for testing purpose), and try asking questions. Using settings in the upper right corner we take apiKey and add it to createAimybox instead of YOUR KEY. private fun createAimybox (context: Context): Aimybox {
        val unitId = UUID.randomUUID().toString()
        val textToSpeech = GooglePlatformTextToSpeech(context)
        val speechToText = GooglePlatformSpeechToText(context)
        val dialogApi = AimyboxDialogApi( , unitId) Aimybox(Config.create(speechToText, textToSpeech, dialogApi))
    } "YOUR KEY" return It works! Here’s a repository link Read the second part Previously published at https://just-ai.com/en/blog/how-to-add-a-voice-assistant-to-your-mobile-app

How to Add a Voice Assistant to your Mobile App?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

How We Built a Voice-Activated Lovecraftian Adventure Game

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

How We Built a Voice-Activated Lovecraftian Adventure Game

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps