Skip to main content

Baosen Zhang

  • Assistant Professor

Appointments

Assistant Professor, Electrical Engineering
Affiliated Faculty, Clean Energy Institute

Biography

Baosen Zhang is an assistant professor in the Department of Electrical Engineering at the University of Washington. He received his B.A.Sc. degree in engineering science from the University of Toronto, Toronto, ON, Canada, in 2008 and his Ph.D. from the Department of Electrical Engineering and Computer Science at the University of California at Berkeley, in 2013. Before joining UW, he was postdoctoral scholar at Stanford University, jointly hosted by departments of Civil and Environmental Engineering and Management & Science Engineering. His interest is in the area of power systems and cyberphysical systems, particularly in the fundamentals of physical resource allocations, economics, and controlling systems with humans in the loop.

Research Interests

Power systems; cyberphysical Systems; smart cities; and optimizing while learning.

2uweeViewNews Object
(
    [_showAnnouncements:protected] => 
    [_showTitle:protected] => 
    [showMore] => 
    [_type:protected] => spotlight
    [_from:protected] => person
    [_args:protected] => Array
        (
            [post_type] => spotlight
            [date_query] => Array
                (
                    [0] => Array
                        (
                            [after] => Array
                                (
                                    [year] => 2015
                                    [month] => 8
                                    [day] => 17
                                )

                        )

                )

            [meta_query] => Array
                (
                    [relation] => AND
                    [0] => Array
                        (
                            [key] => type
                            [value] => news
                            [compare] => LIKE
                        )

                    [1] => Array
                        (
                            [key] => subjects
                            [value] => "920"
                            [compare] => LIKE
                        )

                )

            [posts_per_page] => 6
            [post_status] => publish
        )

    [_jids:protected] => 
    [_taxa:protected] => Array
        (
        )

    [_meta:protected] => Array
        (
            [0] => Array
                (
                    [key] => type
                    [value] => news
                    [compare] => LIKE
                )

            [1] => Array
                (
                    [key] => subjects
                    [value] => "920"
                    [compare] => LIKE
                )

        )

    [_metarelation:protected] => AND
    [_results:protected] => Array
        (
            [0] => WP_Post Object
                (
                    [ID] => 10049
                    [post_author] => 12
                    [post_date] => 2017-02-28 13:24:49
                    [post_date_gmt] => 2017-02-28 21:24:49
                    [post_content] => [caption id="attachment_10052" align="alignleft" width="434"]nsl-perspective_team-photo_2 The UW electrical engineering research team includes (left to right) Professor and Chair Radha Poovendran, doctoral student Hossein Hosseini, Assistant Professor Baosen Zhang and Assistant Professor Sreeram Kannan (not pictured.).[/caption]

University of Washington electrical engineering researchers have shown that Google’s new machine learning-based system to identify toxic comments in online discussion forums can be bypassed by simply misspelling or adding unnecessary punctuation to abusive words, such as “idiot” or “moron.”

Perspective is a project by Google’s technology incubator Jigsaw, which uses artificial intelligence to combat internet trolls and promote more civil online discussion by automatically detecting online insults, harassment and abusive speech.  The company launched a demonstration website on Feb. 23 that allows anyone to type in a phrase and see its “toxicity score” — a measure of how rude, disrespectful or unreasonable a particular comment is.

In a paper posted Feb. 27 on the e-print repository arXiv, the UW electrical engineers and security experts demonstrated that the early stage technology system can be deceived by using common adversarial tactics. They showed one can subtly modify a phrase that receives a high toxicity score so that it contains the same abusive language but receives a low toxicity score.

Given that news platforms such as The New York Times and other media companies are exploring how the system could help curb harassment and abuse in online comment areas or social media, the UW researchers evaluated Perspective in adversarial settings. They showed that the system is vulnerable to both missing incendiary language and falsely blocking non-abusive phrases.

“Machine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,” said senior author Radha Poovendran, chair of the UW electrical engineering department and director of the Network Security Lab. “We wanted to demonstrate the importance of designing these machine learning tools in adversarial environments. Designing a system with a benign operating environment in mind and deploying it in adversarial environments can have devastating consequences.”

To solicit feedback and invite other researchers to explore the strengths and weaknesses of using machine learning as a tool to improve online discussions, Perspective developers made their experiments, models and data publicly available along with the tool itself.

In the examples below on hot-button topics of climate change, Brexit and the recent U.S. election — which were taken directly from the Perspective API website — the UW team simply misspelled or added extraneous punctuation or spaces to the offending words, which yielded much lower toxicity scores. For example, simply changing “idiot” to “idiiot” reduced the toxicity rate of an otherwise identical comment from 84% to 20%.

nsl-google-perspective_graphic-1

In the examples below, the researchers also showed that the system does not assign a low toxicity score to a negated version of an abusive phrase.

nsl-google-perspective_graphic-2

The researchers also observed that the duplicitous changes often transfer among different phrases — once an intentionally misspelled word was given a low toxicity score in one phrase, it was also given a low score in another phrase. That means an adversary could create a “dictionary” of changes for every word and significantly simplify the attack process.

“There are two metrics for evaluating the performance of a filtering system like a spam blocker or toxic speech detector; one is the missed detection rate and the other is the false alarm rate,” said lead author and UW electrical engineering doctoral student Hossein Hosseini. “Of course scoring the semantic toxicity of a phrase is challenging, but deploying defensive mechanisms both in algorithmic and system levels can help the usability of the system in real-world settings.”

The research team suggests several techniques to improve the robustness of toxic speech detectors, including applying a spellchecking filter prior to the detection system, training the machine learning algorithm with adversarial examples and blocking suspicious users for a period of time.

“Our Network Security Lab research is typically focused on the foundations and science of cybersecurity,” said Poovendran, the lead principal investigator of a recently awarded MURI grant, of which adversarial machine learning is a significant component. “But our expanded focus includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.”

Co-authors include UW electrical engineering assistant professors Sreeram Kannan and Baosen Zhang.

The research is funded by the National Science Foundation, the Office of Naval Research and the Army Research Office.

More News:

[post_title] => UW Security Researchers Show that Google’s AI Platform for Defeating Internet Trolls Can be Easily Deceived [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => uw-security-researchers-show-that-googles-ai-platform-for-defeating-internet-trolls-can-be-easily-deceived [to_ping] => [pinged] => [post_modified] => 2017-05-01 16:55:08 [post_modified_gmt] => 2017-05-01 23:55:08 [post_content_filtered] => [post_parent] => 0 [guid] => http://www.ee.washington.edu/?post_type=spotlight&p=10049 [menu_order] => 62 [post_type] => spotlight [post_mime_type] => [comment_count] => 0 [filter] => raw ) [1] => WP_Post Object ( [ID] => 7731 [post_author] => 12 [post_date] => 2016-10-17 16:16:39 [post_date_gmt] => 2016-10-17 23:16:39 [post_content] => [caption id="attachment_7732" align="alignleft" width="377"]lillian-and-baosen EE Assistant Professor Baosen Zhang is PI on the project. EE Assistant Professor Lillian Ratliff is Co-PI.[/caption] For urban roadways, traffic-choked streets have become synonymous with the weekday commute. Over the decades, strategic conversations between city officials, engineers and policy makers have sought to lessen congestion and provide increased transportation options. However, as cities continue to develop and populations increase, the results of years of conversation cannot materialize fast enough. On the thrumming streets of Seattle and Nashville, the consumer becomes a key player on urban transportation initiatives. The project, which is a collaboration between the University of Washington, City of Seattle, Vanderbilt University and the City of Nashville, tackles urban transportation congestion by engaging the individual user through the use of smart devices. The three-year, proof-of-concept project has received a collaborative National Science Foundation (NSF) US Ignite Grant. Electrical Engineering Assistant Professor Baosen Zhang is the Principal Investigator on the project and Electrical Engineering Assistant Professor Lillian Ratliff is the Co-Principal Investigator. The University of Washington leads the multimodal transit project, collaborating with Vanderbilt University and the Cities of Nashville and Seattle to test the research. Zhang and his team seek to build an overarching solution that balances the needs of multiple parties, including commercial companies, municipal service providers and individuals. The information sharing and computing platform overcomes the incentive gap between municipalities and individuals by offering mixed-mode routing suggestions and other relevant information to travelers. For municipality officials, it relays how users are consuming different transportation resources. “The platform serves as a virtual commons,” Ratliff said. “Individual citizens can directly communicate with service providers. It offers beneficial information to these providers, while offering users a voice.” The project utilizes smart devices due to their proliferation in the urban commuter space. The commuters, therefore, become active agents in a shared economy. Currently available applications for multimodal transport solutions focus on individual users and their local perspectives. This current application does not accurately represent an overall solution. Although there is large-scale data being collected by both municipalities and users, neither group has the resources to develop real-time analytics and controls. The project will develop an architecture and framework to perform on a distributed platform and utilize multiple routes. The researchers will also develop the software to host a social platform capable of delivering relevant data and analytics. The Cities of Seattle and Nashville offer real-world use for testing and implementation. “No one has done this type of collaborating and computing before,” Ratliff said. “It not only focuses on commuters as a whole, but it also looks at two socioeconomically diverse cities – Seattle and Nashville.” Through an additional NSF Grant – the Early-Concept Grant for Exploratory Research (EAGER) – Zhang and Ratliff collaborate with the City of Seattle to alleviate parking challenges within the city. This project will address a host of environmental and infrastructure concerns, such as health, the environment and urban development. “Traffic congestions are increasingly becoming bottlenecks to sustainable urban growth as infrastructures are being stretched to their limits,” Zhang said. “Up to 40 percent of all surface level traffic in urban areas stems from drivers looking for parking. This project will develop new parking management tools using algorithms for cities and apps for drivers that allow municipalities to achieve better congestion control and enable drivers to act more efficiently.” The information gathered will provide parking and congestion models to municipalities, allowing the city to achieve better congestion control and enable drivers to act more efficiently. “Transportation is a public good,” Ratliff said. “If this pilot is successful, this will inform how we can engage the citizens more, not just with traffic congestion, but with other transportation and urban initiatives.” These projects are part of the Smart and Connected Communities Initiative of the UW Electrical Engineering Department. The University of Washington is part of the MetroLab Network initiative of the White House. [post_title] => UW EE Faculty to Tackle Urban Mobility [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => electrical-engineering-professors-zhang-and-ratliff-to-develop-novel-multimodal-transportation-project-with-vanderbilt-university-and-the-cities-of-seattle-and-nashville [to_ping] => [pinged] => [post_modified] => 2016-11-02 11:39:08 [post_modified_gmt] => 2016-11-02 18:39:08 [post_content_filtered] => [post_parent] => 0 [guid] => http://www.ee.washington.edu/?post_type=spotlight&p=7731 [menu_order] => 110 [post_type] => spotlight [post_mime_type] => [comment_count] => 0 [filter] => raw ) ) [_numposts:protected] => 6 [_rendered:protected] => 1 [_classes:protected] => Array ( [0] => block--spotlight-tiles ) [_finalHTML:protected] => [_postID:protected] => 920 [_errors:protected] => Array ( ) [_block:protected] => [_db:protected] => WP_Query Object ( [query] => Array ( [post_type] => spotlight [date_query] => Array ( [0] => Array ( [after] => Array ( [year] => 2015 [month] => 8 [day] => 17 ) ) ) [meta_query] => Array ( [relation] => AND [0] => Array ( [key] => type [value] => news [compare] => LIKE ) [1] => Array ( [key] => subjects [value] => "920" [compare] => LIKE ) ) [posts_per_page] => 6 [post_status] => publish ) [query_vars] => Array ( [post_type] => spotlight [date_query] => Array ( [0] => Array ( [after] => Array ( [year] => 2015 [month] => 8 [day] => 17 ) ) ) [meta_query] => Array ( [relation] => AND [0] => Array ( [key] => type [value] => news [compare] => LIKE ) [1] => Array ( [key] => subjects [value] => "920" [compare] => LIKE ) ) [posts_per_page] => 6 [post_status] => publish [error] => [m] => [p] => 0 [post_parent] => [subpost] => [subpost_id] => [attachment] => [attachment_id] => 0 [name] => [static] => [pagename] => [page_id] => 0 [second] => [minute] => [hour] => [day] => 0 [monthnum] => 0 [year] => 0 [w] => 0 [category_name] => [tag] => [cat] => [tag_id] => [author] => [author_name] => [feed] => [tb] => [paged] => 0 [meta_key] => [meta_value] => [preview] => [s] => [sentence] => [title] => [fields] => [menu_order] => [embed] => [category__in] => Array ( ) [category__not_in] => Array ( ) [category__and] => Array ( ) [post__in] => Array ( ) [post__not_in] => Array ( ) [post_name__in] => Array ( ) [tag__in] => Array ( ) [tag__not_in] => Array ( ) [tag__and] => Array ( ) [tag_slug__in] => Array ( ) [tag_slug__and] => Array ( ) [post_parent__in] => Array ( ) [post_parent__not_in] => Array ( ) [author__in] => Array ( ) [author__not_in] => Array ( ) [orderby] => menu_order [order] => ASC [ignore_sticky_posts] => [suppress_filters] => [cache_results] => 1 [update_post_term_cache] => 1 [lazy_load_term_meta] => 1 [update_post_meta_cache] => 1 [nopaging] => [comments_per_page] => 50 [no_found_rows] => ) [tax_query] => WP_Tax_Query Object ( [queries] => Array ( ) [relation] => AND [table_aliases:protected] => Array ( ) [queried_terms] => Array ( ) [primary_table] => wp_posts [primary_id_column] => ID ) [meta_query] => WP_Meta_Query Object ( [queries] => Array ( [0] => Array ( [key] => type [value] => news [compare] => LIKE ) [1] => Array ( [key] => subjects [value] => "920" [compare] => LIKE ) [relation] => AND ) [relation] => AND [meta_table] => wp_postmeta [meta_id_column] => post_id [primary_table] => wp_posts [primary_id_column] => ID [table_aliases:protected] => Array ( [0] => wp_postmeta [1] => mt1 ) [clauses:protected] => Array ( [wp_postmeta] => Array ( [key] => type [value] => news [compare] => LIKE [alias] => wp_postmeta [cast] => CHAR ) [mt1] => Array ( [key] => subjects [value] => "920" [compare] => LIKE [alias] => mt1 [cast] => CHAR ) ) [has_or_relation:protected] => ) [date_query] => WP_Date_Query Object ( [queries] => Array ( [0] => Array ( [after] => Array ( [year] => 2015 [month] => 8 [day] => 17 ) [column] => post_date [compare] => = [relation] => AND ) [column] => post_date [compare] => = [relation] => AND ) [relation] => AND [column] => wp_posts.post_date [compare] => = [time_keys] => Array ( [0] => after [1] => before [2] => year [3] => month [4] => monthnum [5] => week [6] => w [7] => dayofyear [8] => day [9] => dayofweek [10] => dayofweek_iso [11] => hour [12] => minute [13] => second ) ) [request] => SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts INNER JOIN wp_postmeta ON ( wp_posts.ID = wp_postmeta.post_id ) INNER JOIN wp_postmeta AS mt1 ON ( wp_posts.ID = mt1.post_id ) WHERE 1=1 AND ( wp_posts.post_date > '2015-08-17 23:59:59' ) AND ( ( wp_postmeta.meta_key = 'type' AND wp_postmeta.meta_value LIKE '%news%' ) AND ( mt1.meta_key = 'subjects' AND mt1.meta_value LIKE '%\"920\"%' ) ) AND wp_posts.post_type = 'spotlight' AND ((wp_posts.post_status = 'publish')) GROUP BY wp_posts.ID ORDER BY wp_posts.menu_order ASC LIMIT 0, 6 [posts] => Array ( [0] => WP_Post Object ( [ID] => 10049 [post_author] => 12 [post_date] => 2017-02-28 13:24:49 [post_date_gmt] => 2017-02-28 21:24:49 [post_content] => [caption id="attachment_10052" align="alignleft" width="434"]nsl-perspective_team-photo_2 The UW electrical engineering research team includes (left to right) Professor and Chair Radha Poovendran, doctoral student Hossein Hosseini, Assistant Professor Baosen Zhang and Assistant Professor Sreeram Kannan (not pictured.).[/caption]

University of Washington electrical engineering researchers have shown that Google’s new machine learning-based system to identify toxic comments in online discussion forums can be bypassed by simply misspelling or adding unnecessary punctuation to abusive words, such as “idiot” or “moron.”

Perspective is a project by Google’s technology incubator Jigsaw, which uses artificial intelligence to combat internet trolls and promote more civil online discussion by automatically detecting online insults, harassment and abusive speech.  The company launched a demonstration website on Feb. 23 that allows anyone to type in a phrase and see its “toxicity score” — a measure of how rude, disrespectful or unreasonable a particular comment is.

In a paper posted Feb. 27 on the e-print repository arXiv, the UW electrical engineers and security experts demonstrated that the early stage technology system can be deceived by using common adversarial tactics. They showed one can subtly modify a phrase that receives a high toxicity score so that it contains the same abusive language but receives a low toxicity score.

Given that news platforms such as The New York Times and other media companies are exploring how the system could help curb harassment and abuse in online comment areas or social media, the UW researchers evaluated Perspective in adversarial settings. They showed that the system is vulnerable to both missing incendiary language and falsely blocking non-abusive phrases.

“Machine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,” said senior author Radha Poovendran, chair of the UW electrical engineering department and director of the Network Security Lab. “We wanted to demonstrate the importance of designing these machine learning tools in adversarial environments. Designing a system with a benign operating environment in mind and deploying it in adversarial environments can have devastating consequences.”

To solicit feedback and invite other researchers to explore the strengths and weaknesses of using machine learning as a tool to improve online discussions, Perspective developers made their experiments, models and data publicly available along with the tool itself.

In the examples below on hot-button topics of climate change, Brexit and the recent U.S. election — which were taken directly from the Perspective API website — the UW team simply misspelled or added extraneous punctuation or spaces to the offending words, which yielded much lower toxicity scores. For example, simply changing “idiot” to “idiiot” reduced the toxicity rate of an otherwise identical comment from 84% to 20%.

nsl-google-perspective_graphic-1

In the examples below, the researchers also showed that the system does not assign a low toxicity score to a negated version of an abusive phrase.

nsl-google-perspective_graphic-2

The researchers also observed that the duplicitous changes often transfer among different phrases — once an intentionally misspelled word was given a low toxicity score in one phrase, it was also given a low score in another phrase. That means an adversary could create a “dictionary” of changes for every word and significantly simplify the attack process.

“There are two metrics for evaluating the performance of a filtering system like a spam blocker or toxic speech detector; one is the missed detection rate and the other is the false alarm rate,” said lead author and UW electrical engineering doctoral student Hossein Hosseini. “Of course scoring the semantic toxicity of a phrase is challenging, but deploying defensive mechanisms both in algorithmic and system levels can help the usability of the system in real-world settings.”

The research team suggests several techniques to improve the robustness of toxic speech detectors, including applying a spellchecking filter prior to the detection system, training the machine learning algorithm with adversarial examples and blocking suspicious users for a period of time.

“Our Network Security Lab research is typically focused on the foundations and science of cybersecurity,” said Poovendran, the lead principal investigator of a recently awarded MURI grant, of which adversarial machine learning is a significant component. “But our expanded focus includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.”

Co-authors include UW electrical engineering assistant professors Sreeram Kannan and Baosen Zhang.

The research is funded by the National Science Foundation, the Office of Naval Research and the Army Research Office.

More News:

[post_title] => UW Security Researchers Show that Google’s AI Platform for Defeating Internet Trolls Can be Easily Deceived [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => uw-security-researchers-show-that-googles-ai-platform-for-defeating-internet-trolls-can-be-easily-deceived [to_ping] => [pinged] => [post_modified] => 2017-05-01 16:55:08 [post_modified_gmt] => 2017-05-01 23:55:08 [post_content_filtered] => [post_parent] => 0 [guid] => http://www.ee.washington.edu/?post_type=spotlight&p=10049 [menu_order] => 62 [post_type] => spotlight [post_mime_type] => [comment_count] => 0 [filter] => raw ) [1] => WP_Post Object ( [ID] => 7731 [post_author] => 12 [post_date] => 2016-10-17 16:16:39 [post_date_gmt] => 2016-10-17 23:16:39 [post_content] => [caption id="attachment_7732" align="alignleft" width="377"]lillian-and-baosen EE Assistant Professor Baosen Zhang is PI on the project. EE Assistant Professor Lillian Ratliff is Co-PI.[/caption] For urban roadways, traffic-choked streets have become synonymous with the weekday commute. Over the decades, strategic conversations between city officials, engineers and policy makers have sought to lessen congestion and provide increased transportation options. However, as cities continue to develop and populations increase, the results of years of conversation cannot materialize fast enough. On the thrumming streets of Seattle and Nashville, the consumer becomes a key player on urban transportation initiatives. The project, which is a collaboration between the University of Washington, City of Seattle, Vanderbilt University and the City of Nashville, tackles urban transportation congestion by engaging the individual user through the use of smart devices. The three-year, proof-of-concept project has received a collaborative National Science Foundation (NSF) US Ignite Grant. Electrical Engineering Assistant Professor Baosen Zhang is the Principal Investigator on the project and Electrical Engineering Assistant Professor Lillian Ratliff is the Co-Principal Investigator. The University of Washington leads the multimodal transit project, collaborating with Vanderbilt University and the Cities of Nashville and Seattle to test the research. Zhang and his team seek to build an overarching solution that balances the needs of multiple parties, including commercial companies, municipal service providers and individuals. The information sharing and computing platform overcomes the incentive gap between municipalities and individuals by offering mixed-mode routing suggestions and other relevant information to travelers. For municipality officials, it relays how users are consuming different transportation resources. “The platform serves as a virtual commons,” Ratliff said. “Individual citizens can directly communicate with service providers. It offers beneficial information to these providers, while offering users a voice.” The project utilizes smart devices due to their proliferation in the urban commuter space. The commuters, therefore, become active agents in a shared economy. Currently available applications for multimodal transport solutions focus on individual users and their local perspectives. This current application does not accurately represent an overall solution. Although there is large-scale data being collected by both municipalities and users, neither group has the resources to develop real-time analytics and controls. The project will develop an architecture and framework to perform on a distributed platform and utilize multiple routes. The researchers will also develop the software to host a social platform capable of delivering relevant data and analytics. The Cities of Seattle and Nashville offer real-world use for testing and implementation. “No one has done this type of collaborating and computing before,” Ratliff said. “It not only focuses on commuters as a whole, but it also looks at two socioeconomically diverse cities – Seattle and Nashville.” Through an additional NSF Grant – the Early-Concept Grant for Exploratory Research (EAGER) – Zhang and Ratliff collaborate with the City of Seattle to alleviate parking challenges within the city. This project will address a host of environmental and infrastructure concerns, such as health, the environment and urban development. “Traffic congestions are increasingly becoming bottlenecks to sustainable urban growth as infrastructures are being stretched to their limits,” Zhang said. “Up to 40 percent of all surface level traffic in urban areas stems from drivers looking for parking. This project will develop new parking management tools using algorithms for cities and apps for drivers that allow municipalities to achieve better congestion control and enable drivers to act more efficiently.” The information gathered will provide parking and congestion models to municipalities, allowing the city to achieve better congestion control and enable drivers to act more efficiently. “Transportation is a public good,” Ratliff said. “If this pilot is successful, this will inform how we can engage the citizens more, not just with traffic congestion, but with other transportation and urban initiatives.” These projects are part of the Smart and Connected Communities Initiative of the UW Electrical Engineering Department. The University of Washington is part of the MetroLab Network initiative of the White House. [post_title] => UW EE Faculty to Tackle Urban Mobility [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => electrical-engineering-professors-zhang-and-ratliff-to-develop-novel-multimodal-transportation-project-with-vanderbilt-university-and-the-cities-of-seattle-and-nashville [to_ping] => [pinged] => [post_modified] => 2016-11-02 11:39:08 [post_modified_gmt] => 2016-11-02 18:39:08 [post_content_filtered] => [post_parent] => 0 [guid] => http://www.ee.washington.edu/?post_type=spotlight&p=7731 [menu_order] => 110 [post_type] => spotlight [post_mime_type] => [comment_count] => 0 [filter] => raw ) ) [post_count] => 2 [current_post] => -1 [in_the_loop] => [post] => WP_Post Object ( [ID] => 10049 [post_author] => 12 [post_date] => 2017-02-28 13:24:49 [post_date_gmt] => 2017-02-28 21:24:49 [post_content] => [caption id="attachment_10052" align="alignleft" width="434"]nsl-perspective_team-photo_2 The UW electrical engineering research team includes (left to right) Professor and Chair Radha Poovendran, doctoral student Hossein Hosseini, Assistant Professor Baosen Zhang and Assistant Professor Sreeram Kannan (not pictured.).[/caption]

University of Washington electrical engineering researchers have shown that Google’s new machine learning-based system to identify toxic comments in online discussion forums can be bypassed by simply misspelling or adding unnecessary punctuation to abusive words, such as “idiot” or “moron.”

Perspective is a project by Google’s technology incubator Jigsaw, which uses artificial intelligence to combat internet trolls and promote more civil online discussion by automatically detecting online insults, harassment and abusive speech.  The company launched a demonstration website on Feb. 23 that allows anyone to type in a phrase and see its “toxicity score” — a measure of how rude, disrespectful or unreasonable a particular comment is.

In a paper posted Feb. 27 on the e-print repository arXiv, the UW electrical engineers and security experts demonstrated that the early stage technology system can be deceived by using common adversarial tactics. They showed one can subtly modify a phrase that receives a high toxicity score so that it contains the same abusive language but receives a low toxicity score.

Given that news platforms such as The New York Times and other media companies are exploring how the system could help curb harassment and abuse in online comment areas or social media, the UW researchers evaluated Perspective in adversarial settings. They showed that the system is vulnerable to both missing incendiary language and falsely blocking non-abusive phrases.

“Machine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,” said senior author Radha Poovendran, chair of the UW electrical engineering department and director of the Network Security Lab. “We wanted to demonstrate the importance of designing these machine learning tools in adversarial environments. Designing a system with a benign operating environment in mind and deploying it in adversarial environments can have devastating consequences.”

To solicit feedback and invite other researchers to explore the strengths and weaknesses of using machine learning as a tool to improve online discussions, Perspective developers made their experiments, models and data publicly available along with the tool itself.

In the examples below on hot-button topics of climate change, Brexit and the recent U.S. election — which were taken directly from the Perspective API website — the UW team simply misspelled or added extraneous punctuation or spaces to the offending words, which yielded much lower toxicity scores. For example, simply changing “idiot” to “idiiot” reduced the toxicity rate of an otherwise identical comment from 84% to 20%.

nsl-google-perspective_graphic-1

In the examples below, the researchers also showed that the system does not assign a low toxicity score to a negated version of an abusive phrase.

nsl-google-perspective_graphic-2

The researchers also observed that the duplicitous changes often transfer among different phrases — once an intentionally misspelled word was given a low toxicity score in one phrase, it was also given a low score in another phrase. That means an adversary could create a “dictionary” of changes for every word and significantly simplify the attack process.

“There are two metrics for evaluating the performance of a filtering system like a spam blocker or toxic speech detector; one is the missed detection rate and the other is the false alarm rate,” said lead author and UW electrical engineering doctoral student Hossein Hosseini. “Of course scoring the semantic toxicity of a phrase is challenging, but deploying defensive mechanisms both in algorithmic and system levels can help the usability of the system in real-world settings.”

The research team suggests several techniques to improve the robustness of toxic speech detectors, including applying a spellchecking filter prior to the detection system, training the machine learning algorithm with adversarial examples and blocking suspicious users for a period of time.

“Our Network Security Lab research is typically focused on the foundations and science of cybersecurity,” said Poovendran, the lead principal investigator of a recently awarded MURI grant, of which adversarial machine learning is a significant component. “But our expanded focus includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.”

Co-authors include UW electrical engineering assistant professors Sreeram Kannan and Baosen Zhang.

The research is funded by the National Science Foundation, the Office of Naval Research and the Army Research Office.

More News:

[post_title] => UW Security Researchers Show that Google’s AI Platform for Defeating Internet Trolls Can be Easily Deceived [post_excerpt] => [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => uw-security-researchers-show-that-googles-ai-platform-for-defeating-internet-trolls-can-be-easily-deceived [to_ping] => [pinged] => [post_modified] => 2017-05-01 16:55:08 [post_modified_gmt] => 2017-05-01 23:55:08 [post_content_filtered] => [post_parent] => 0 [guid] => http://www.ee.washington.edu/?post_type=spotlight&p=10049 [menu_order] => 62 [post_type] => spotlight [post_mime_type] => [comment_count] => 0 [filter] => raw ) [comment_count] => 0 [current_comment] => -1 [found_posts] => 2 [max_num_pages] => 1 [max_num_comment_pages] => 0 [is_single] => [is_preview] => [is_page] => [is_archive] => 1 [is_date] => [is_year] => [is_month] => [is_day] => [is_time] => [is_author] => [is_category] => [is_tag] => [is_tax] => [is_search] => [is_feed] => [is_comment_feed] => [is_trackback] => [is_home] => [is_404] => [is_embed] => [is_paged] => [is_admin] => [is_attachment] => [is_singular] => [is_robots] => [is_posts_page] => [is_post_type_archive] => 1 [query_vars_hash:WP_Query:private] => 23ee11ba76ad10d43a631f062acc425d [query_vars_changed:WP_Query:private] => 1 [thumbnails_cached] => [stopwords:WP_Query:private] => [compat_fields:WP_Query:private] => Array ( [0] => query_vars_hash [1] => query_vars_changed ) [compat_methods:WP_Query:private] => Array ( [0] => init_query_flags [1] => parse_tax_query ) ) )
 

Representative Publications

  • B. Zhang, A. Lam, A. Dominguez-Garcia and D. Tse, "An Optimal and Distributed Method for Voltage Regulation in Power Distribution Systems", IEEE Trans. Power Syst., vol. 30, no. 4, pp. 1714-1726, 2015.
  • C. Riquelme, R. Johari and B. Zhang, "Online Active Linear Regression via Thresholding", ArXiv, 2016.
  • B. Zhang, R. Johari and R. Rajagopal, "Competition and Coalition Formation of Renewable Power Producers", IEEE Trans. Power Syst., vol. 30, no. 3, pp. 1624-1632, 2015.
  • B. Zhang and D. Tse, "Geometry of injection regions of power networks", IEEE Trans. Power Syst., vol. 28, no. 2, pp. 788-797, 2013.

Research Areas

Affiliations

Education

  • Ph.D., Electrical Engineering and Computer Science, 2013
    University of California, Berkeley
  • B.A.Sc., Engineering Science (Electrical Option)
    University of Toronto, 2008