Develop Accessibility Support for Instant Messaging App

How to make Alibaba’s DingTalk Android accessible for visually impaired For the millions of people around the world living with visual impairment or blindness, using a smartphone or tablet can prove challenging. It’s essential that we make applications accessible for as many users as possible, and Alibaba engineers have been working to make DingTalk, Alibaba’s powerful instant messaging tool for enterprises, accessible to the visually impaired. There are about 285 million people in the world living with a visual impairment, but with the help of smart devices they are gaining access to the internet and all the benefits that brings. But in a world where most interactions are carried out online, not being able to read what’s on their screen can prevent visually impaired people from realizing the full benefits of the internet as much as those with full visual acuity. In June 2017, to complement the accessibility support already in place for DingTalk iOS, Alibaba launched a set of accessibility improvements for DingTalk Android. Having worked closely with the Information Accessibility Research Association in Shenzhen for over two months, DingTalk Android is now fully accessible and user-friendly. It is currently already in use at various organizations and institutions, including the Zhejiang School for the Blind. What is accessibility support? Visually impaired users often need to use screen-reading software to know what’s on their screens and they mainly rely on their hearing to complete interactions. For example, Android devices use TalkBack, which uses spoken-words, vibrations and other audible feedback to let blind users know what is on their screen, what they’re touching and what their options are. It also introduces several gestures that allow the visually impaired to navigate through the different options available to them. Screenshots of the newly accessible DingTalk, showing the tasks that can be performed with accessibility support For example, when the ‘Text-to-speech output’ option is enabled, any text can be read aloud. All that is required is to set the focus to a specific view and let the screen reader read out the most suitable descriptions. This enables blind users to know exactly where they are and what they can do next. Accessibility event distribution To better understand how the Android accessibility framework works, and therefore provide the best possible accessibility support, the DingTalk team analyze accessibility event distribution from the source level. sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_SELECTED); sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_CLICKED); In View.java, various method calls similar to the sendAccessibilityEvent above are displayed. This is the starting point for accessibility event distribution: The onAccessibilityEvent (AccessibilityEvent event) method of the screen-reading app, which implements the AccessibilityService, is then called to complete the accessibility prompt. The image above shows the process of accessibility event distribution from the method call order. Accessibility work list There are five key areas that should be addressed when developing accessibility support for an app or other software. 1) ContentDescription label for elements This is the easiest and most important part of accessibility support. Simply put, this determines the description the user hears, when different elements or controls are navigated to, and constitutes up to 80% of accessibility support work. One thing to bear in mind is that contentDescription needs to be clear and concise. Lengthy description texts will affect operational efficiency for blind users. TalkBack uses the following attributes of the View component: · contentDescription · Component type (image, button and so on) · Status (such as CheckBox checked). These are automatically supported by the Android framework. For ImageView, ImageButton and CheckBox, contentDescription is an attribute that must be added. TextView, Button and other elements with the text attribute can be selected without being added, and the Android framework will use its text attribute as the screen-reading content. 2) Start focus navigation, start view focus and control the focus order As visually impaired users cannot accurately identify the location of operable elements in the interface, they instead need to switch the focus between different operable elements by swiping left and right. Another obstacle therefore is to manage the focus of elements. The elements the Android framework provides are focusable by default. Android also provides APIs that allow developers both to determine whether elements can be focused, as well as to request focus for elements. We should ensure that each operable element can be focused. However, elements that interfere with accessible users’ user experience should not be set to receive focus. The focus order is based on an algorithm that looks for adjacent elements in a particular direction. In some cases, the default algorithm may not match the developer-defined order or may not be logical to the user. In these cases, the focus order can be explicitly overridden with the android:nextFocusxxx attribute in the layout file. 3) Customized view If an accessibility event provided by the Android framework doesn’t meet particular needs, for example you want to feed the current progress back to the visually impaired user when they drag a progress bar, then the view needs to be customized. The source code of the accessibility event distribution process shown above, gives a rough idea of what a customized view can do in terms of accessibility support. sendAccessibilityEvent() // Send an accessible event at a suitable time onInitializeAccessibilityEvent() // Initialize an accessibility event dispatchPopulateAccessibilityEvent() // Rewrite an accessibility event onInitializeAccessibilityNodeInfo() // This method fills the AccessibilityNodeInfo object, which provides more context for the accessibility services, and provides the user with the appropriate feedback. For further details, visit the Customized view section at . http://www.siaa.org.cn/androidguideline.pdf Android provides standard accessibility services, including TalkBack, but developers can also create and publish their own. Accessibility services can even be used in lieu of user operations to perform a number of advanced tech functions. 4. Customized accessibility services Take the Google’s sample demo as an example. Accessibility statements, like any other service, need to be declared in AndroidManifest.xml. The configuration of accessible service parameters can be set dynamically as shown above, and can also use tags after android 4.0, as below. This is the content of the file of taskbackconfig.xml referred to in AndroidManifest.xml. Several important labels are described below. _accessibilityEventTypes_ //Specifies the type of time to monitor, such as clicks or window changes _packageNames_ //Specifies the name of the application package that this accessibility service can handle, with multiple applications separated by a comma (,) _canRetrieveWindowContent_ //Specifies whether you can get to the contents of the window Customized accessibility services need to implement AccessibilityService and override these methods as onAccessibilityEvent, onInterrupt and so on. Information, such as View context, can be obtained through AccessibilityEvent, and can even do things in lieu of users. Finally, there is also access to an AccessibilityEvent to perform audio or vibration feedback, thereby providing accessibility services. 5) Tests It takes time to make applications fully accessible, especially ones like DingTalk which have multiple features that can be missed or handled incorrectly. It’s also important to prevent NullPointerExceptions from being introduced when adding contentDescription attributes. Consequently, testing and regression testing are important steps in the process of optimizing accessibility support. Automated testing can be implemented with Espresso or Robolectric. You can also contact a specialized accessibility team for help with acceptance testing. For details on testing methods, go to . https://developer.android.com/training/accessibility/testing.html Additional tips for developing your accessibility support: 1. Conditions permitting, use Android’s built-in interface controls, which provide accessibility support by default. 2. In DingTalk, there is a way to partly use ImageView instead of CheckBox. There is no difference in the UI, but in interactive accessibility mode, blind users won’t be able to correctly determine how the current View is being operated and checked. You should not use ImageView to directly complete CheckBox functions. 3. EditView: Use hint instead of contentDescription. If the text box is empty, hint prompts blind users on what to enter. If it isn’t empty, TalkBack reads the contents of the current input instead of hint. This is lost if you set contentDescription. 4. Small control groups: If the control is smaller than the recommended touch size, consider using the ViewGroup to group the controls together, using android:contentDescription to provide the content description for the combination. To increase the clickable area, reduce the invalid focus. 5. Function change control: If, during the standard operation of an application, a user changes the functionality of a button or other control (for example, changing a button from play to pause), the button’s android:contentDescription needs to be changed accordingly. 6. Associated controls: A set of controls that provide independent functionality, such as a date selector (DatePicker), which provide effective audible feedback when a user interacts with an individual control in an associated control. 7. Additional audible feedback: For example, developers wanting to tell users what the application is doing, such as automatic page turning in a book, use the announceForAccessibility(CharSequence) method to allow the accessibility service to read the information to the user. 8. The contentDescription attribute is the simplest and most efficient way to ensure that your introduction is accurate. 9. Before creating the layout, review and follow the accessibility guidelines provided in the design guide: https://material.io/guidelines/usability/accessibility.html#accessibility-principles Conclusion It’s hard today to imagine a life without the Internet. Accessibility programs help the visually impaired stay connected with the world, and the now fully accessible DingTalk will help visually impaired users both in China and abroad. Alibaba promotes accessibility across the board — making PCs more accessible is currently in development — and as such it will also continue to optimize the accessibility of DingTalk Android and iOS, providing visually impaired users with an enhanced, user-friendly experience. Reference materials http://www.siaa.org.cn/androidguideline.pdf http://informationaccessibilityassociation.github.io/androidAccessibility/services.htm http://www.jianshu.com/p/4cd8c109cdfb https://github.com/appium/android-apidemos/blob/master/src/io/appium/android/apis/accessibility/ClockBackService.java https://darkness463.github.io/2017/04/17/accessibility-event http://informationaccessibilityassociation.github.io/androidAccessibility/checklist.htm (Original article by Chen Pu陈朴) Alibaba Tech First hand and in-depth information about Alibaba’s latest technology → Search on “Alibaba Tech” Facebook