Be Smart and Learn About Your Pictures with JAWS and Fusion!

Introduction.

Within the March 2019 JAWS for Windows and Fusion updates, a new feature is available called Picture Smart. This has the promise to be fantastic technology. Currently as suggested above it is in its early stages of development, but nevertheless it is a useful feature.

We haven't seen much information at all regarding Picture Smart, and there is no help topic available in JAWS or Fusion to support it. So lets walk you through it.

So what is it?

Picture Smart allows you to choose an image on your computer and submit it over the internet to be analyzed. A description of the image is then displayed in the JAWS Results Viewer window. This is a window which overlays the current application or environment in which you are working, allowing you to examine the text without causing any disturbance to your normal working condition. It's the more modern version of the JAWS Virtual Viewer.

Descriptions of pictures might include details such as the number of people in the photo, recognizable landmarks, landscape details such as "outdoors with a blue sky and green grass," if there are vehicles such as cars or trains, and more. If there is text in the picture, it will scan the text using Optical Character Recognition technology and provide that as well.

How Do I Use It?

The following is a list of keystrokes found within the JAWS or Fusion products, together with voice commands which can be used with our J-Say product as of 15 April this year.

To use Picture Smart, press INSERT+SPACEBAR followed by P. JAWS says "Picture Smart". Once the Picture Smart keyboard layer is active, the following keystrokes are available. J-Say users of course do not need to press Insert Space then P.

• A - Open the Picture Smart dialog box where you can acquire an image from a flatbed scanner or the Freedom Scientific PEARL camera. Voice command: "Picture with Camera".
• F - Describes the currently selected image in File Explorer. Supported images include BMP, JPEG, GIF, TIF, or PNG. Voice command: "Picture with File". Note that you must have located an image on your computer first, such as within File Explorer, prior to executing this command or keystroke.
• C - Describes an image associated with the current control. For instance, a graphic on a web page or in an email message. Voice command: "Picture with Control". Note in our experience this is a feature which does not work particularly well at present. If it finds a picture at all, often surrounding text on the page is encapsulated within the description.
• B - Describes the contents of the Windows Clipboard if it contains an image. Voice command: "Picture with Clipboard".
• QUESTION MARK - Displays help for the Picture Smart layer.

There is also a new PictureSmart folder where you can save an image and have it automatically submitted for analyzing and display the resulting description. To access this folder, press WINDOWS Key+E to open File Explorer, open the Pictures folder, and then open the PictureSmart folder.

It is our suggestion that this should be placed on the JAWS Utilities menu for easier access. While JAWS does a good job of monitoring this folder for changes, the only way of examining the description at a later date is to remove the picture and place it back in the folder.

What Kind of Information Does Picture Smart Give You?

Doubtless the information returned by Picture Smart will be refined over time. However, because the results are being collated by using two scanning recognition services, it could be that you will experience some duplication of information. Our view however is that it is better to have duplicated information than none at all.

Prior to scanning a picture, JAWS will advise you that "Picture Smart is in Progress".

Here is a sample of what you might expect, taken from the JAWS Results Viewer.

Caption is a man holding a golf club.
This object appears in the photo

Person.

This object probably appears in the photo

Man.

These tags describe the photo

athletic game, golf, Golf club, Golf course, Golf equipment, Golfer, grass, Grassland, Highland, mountain, outdoor, Pitch and putt, Professional golfer, sky, sport, Sport venue.

This tag probably describes the photo

person.

If there are famous people which form part of the image, JAWS may return the name of that person. I'm obviously not famous at all as it doesn't recognise me! But nor should it.

Conclusion.

To use Picture Smart, the "submit annonymous usage data" option must be enabled in the JAWS Settings Center. This sends encrypted information to Vispero. The images from Picture Smart are then submitted to the various services (such as Microsoft and Google) which analyze the images and return the descriptions back to them. As has been described, these descriptions are then displayed in the Results Viewer. If you attempt to use Picture Smart and this setting is off, a message displays prompting you to turn it on. Simply press Tab to reach the "Yes" Button and press the Space Bar.

Note that the "Anonymous Usage Data" sending is also controllable via the "Startup Wizard" when JAWS or Fusion is first installed, or it can be reached via the JAWS Window by pressing ALT+H then Z.

We do hope this technology continues to evolve, particularly in the area of describing images on web sites such as Facebook. For those people who purchased our thorough training course in the use of Microsoft PowerPoint, an audio demonstration of this feature together with comments are available as part of Lesson 5. Picture Smart is of course closely associated with PowerPoint, since you can verify your pictures prior to placing them into presentations.

We hope you have found this blog post useful! If you would like to find out more about our archived online Training Courses, or on site training within the UK, please visit our Training page!