LEARNER: Bias in Machine Learning

Inclusive Design Research Centre

wecount@inclusivedesign.ca

The Inclusive Design Research Centre is an international community of open source developers, designers, researchers, educators and co-designers who work together to proactively ensure that emerging technology and practices are designed inclusively. Learn more at idrc.ocadu.ca

Tags:

Artificial-Intelligence, Bias, EDI-principles, Machine-Learning, WeCount, Word-Embedding

The earner of this badge has learned how predictive algorithms and data mining affect different populations in a discriminatory manner. In addition, the earner has explored how specific data resources are used to train and reinforce machine learning models to produce biased outputs. The learner viewed a webinar on data bias delivered by Dr. Toon Calders, specialist in machine learning from University of Antwerp. A summative assessment ( knowledge recall quiz including distinguishing terminology and frameworks, and relevant use) was completed and then evaluated by a qualified We Count assessor. Estimated learning time including assessment: 3 hours.

For more information, please visit We Count’s Home Page.

Instructions for Badge Earners

This assessment includes several multiple-choice questions based on the content of Dr. Toon Calders’s webinar on Machine Learning Bias. The second half of the assessment includes a reflection and short answer component to connect what you have learned to We Count initiatives.

This assessment is not time-restricted, featuring an option at the bottom of the form for it to be saved and continued later. For the multiple-choice, please be aware that CanCred selects the first option for each question to be the default answer. Remember to read through all of the questions carefully, and to take your time.

The average learning time for this badge is 3 hours.

Your name *

Your email address *

Part One: Multiple Choice

1. What is Dr. Calders’s definition of machine learning discrimination?

Learning models may unintentionally pickup and amplify bias from training data affected by faulty assumptions, lack of testing and inconsistent sample sizes.

Training data itself is not necessarily biased, but when filtered through a rigid learning procedure, it will inevitably produce discriminative outcomes regardless of the source.

Bias occurs when training data and its outputs reflects reality in an objective way.

Review Webinar if needed.

2. What are the disadvantages of word embeddings being learned from example documents?

If an example document, such as a news article, is neutral in its language, it will exaggerate rather than mitigate biased outcomes from a learning algorithm.

Semantic information inferred from documents can capture and reinforce cultural bias in test data, potentially resulting in discriminatory practices and policies.

Example documents may be highly accurate and balanced in their composition, yet the word embedding will still produce biased outputs.

Review Webinar if needed.

3. How was discrimination incorporated into the COMPAS software (Correctional Offender Management Profiling for Alternative Sanctions) by Northpointe?

The algorithm was twice as likely to label white defenders as high risk who eventually did not re-offend, as compared to black defenders.

There was no discrimination, and the software was treating both parties equally based on the training data it had.

The algorithm was twice as likely to label black defenders as high risk who eventually did not re-offend, as compared to white defenders.

Review Webinar if needed.

4. Based on the webinar, what type of biases can word embedding methods, such as Word2Vec, reinforce from text data that they are trained on?

Algorithmic biases.

Human biases.

Grammatical errors.

Review Webinar if needed.

Part Two: Short Answer and Reflection

Based on what you have learned about sources of bias in the webinar, give 1-2 new or webinar-related examples of possible data bias against minorities and/or persons with disabilities. *

Short paragraph to complete (Max 100 words). We encourage you to share your own examples of Data Bias. Alternatively, feel free to review or expand on examples from the Webinar if needed.

Select one of your examples. *

Put in the name of your example.

From a scale of 1 to 5, with 1 being not important and 5 being very important, how important is it to address this bias in the immediate future?

1 (Not Important)

2 (Slightly important)

3 (Important)

4 (Fairly important)

5 ( Very Important)

Please select one of the options from this scale.

Please explain your reasoning.

Why do you think it is important to address this bias in the immediate future? This section is optional.

I have read the Privacy Notice and I accept it.

- or -

PRIVACY AND COOKIE POLICY
-------------------------

Last Updated June 25, 2020

INTRODUCTION

We ask you to read this privacy and cookie policy ("Privacy Policy")
carefully before using the CanCred Factory service which
is a service issuing and managing digital credentials
("Service").

Commitment to Privacy. Learning Agents Inc., a Canadian
corporation, is the provider of the Service ("Service Provider")
and is committed to privacy by complying with Canada's federal
Personal Information Protection and Electronic Documents Act
("PIPEDA"), Ontario's Freedom of Information and
Protection of Privacy Act ("FIPPA"), British Columbia's
Freedom of Information and Protection of Privacy Act
("FOIPPA") and the European Union's General Data
Protection Regulation ("GDPR").

Definition of Personal Information or Personal Data. Personal
information or personal data means any information about an identified or
identifiable individual.

Content of this Privacy Policy. This Privacy Policy describes
the types of personal information that the users of the Service
("User" or "Users") provide to the Service
Provider or the Service Provider is collecting from the Users in connection
with the Service and how and why the Service Provider collects, uses, discloses
and protects such personal information and the Users' privacy rights in
relation to personal data. In addition, this Privacy Policy outlines how to
contact the Service Provider and supervisory authorities in the event a User
would like to report a concern about the way in which the Service Provider
processes personal information.

The Service. The Service strives to offer all Users a safe
environment to issue and manage digital credentials ("Badges")
based on the Mozilla Open Badge standard. Users operate role-based logins
(e.g. administrator, creator or issuer) on the Service to issue Badges and
manage their accounts. The Service is designed to enable subscribers to the
Service ("Issuers" or "Users") to issue Badges to individuals
who meet prescribed requirements ("Badge Earners").

Designations under GDPR. For the purposes of the GDPR, the
Service Provider is the "processor" of the Badge Earners' personal data acting
on behalf of and based on the instructions of the Users/Issuers, who are the
"controllers" of the Badge Earners' personal data.

Consent. Any User's use of the Service is subject to this
Privacy Policy. If a User does not consent to the processing of his/her
personal information as outlined herein, the Service Provider asks the User to
not use the Service.

Legal Basis in Canada. The Users' use of the Service
constitutes consent to the Service Provider's collection, use and disclosure of
personal information in accordance with this Privacy Policy. No law requires
the User to provide the Service Provider with personal information.

Legal Basis in the European Union. For the applicable various
legal bases of processing in the European Union please refer to the
"DESCRIPTION OF PERSONAL INFORMATION, METHODS FOR ITS COLLECTION AND THE
APPLICABLE LEGAL BASES" section of this Privacy Policy.

Children under the Age of 16. Individuals under the age of 16
are not permitted to use the Service or the Service website. The Service does
not intentionally collect personal information from individuals under the age
of 16. If the Service Provider becomes aware that it has inadvertently
collected personal information about children under the age of 16, it will take
steps to delete the information as soon as possible.

WITHDRAWAL OF USER CONSENT

Users may withdraw their consent to process personal information at any time by
deleting their own accounts and all personal data associated with their
accounts or by contacting the Service Provider to delete their accounts and all
personal data associated with their accounts, provided such withdrawal is
subject to legal or contractual restrictions. If the User withdraws consent,
the Service Provider's ability to provide Services to the User may be
restricted or rendered impossible. The Service Provider will delete the
personal information following the withdrawal of consent, however, such
withdrawal will not affect the lawfulness of processing prior to withdrawal.

DESCRIPTION OF PERSONAL INFORMATION, METHODS FOR ITS COLLECTION AND THE APPLICABLE LEGAL BASES

"CONDITIONS OF SERVICE" INFORMATION. The Service collects the
following information that are necessary and integral to the provision of the
Service, some of which are considered personal information in certain
jurisdictions, but not in others:

- Names of registered Users
- Email addresses of registered Users
- The role of the registered Users
- Names of User organizations
- Website addresses of User organizations
- Organization emails of User organizations
- Mailing addresses of User organizations
- Payment information and payment history of User organizations
- All communication, correspondence and contract history in specific matters
(e.g. testimonials, service reviews, other statements)

Legal Basis in the European Union. It is the obligation of
the Users who are the controllers to determine the legal basis of
processing the personal data of Badge Earners. The processing of personal
data is necessary for the Service Provider to perform the Service and the
related promises and obligations based on the contract entered into between
the User and the Service Provider or if it required for the Service
Provider to comply with any legal obligations

OPTIONAL INFORMATION. The Service also collects the following
information if provided by the User organization:

- Country (and province if in Canada)
- Organization type
- Description of the organization
- Logo and banner graphics

BADGE EARNERS INFORMATION.
The Service also collects the following personal information:

- Email addresses of recipients of Badges ("Badge Earners")
that have been issued by the User/Issuer, as the Badges contain the Badge
Earners' email addresses, phone numbers and URLs in hashed form, which the
User/Issuer has gathered. The User/Issuer acknowledges responsibility for the
collection and safeguarding of the personal information of Badge Earners.

- Other personal information in the Badge may include links to the Badge
Earner's evidence and "Additional Criteria", which can describe how an
individual Badge Earner achieved a particular Badge.

Legal Basis in the European Union. It is the obligation of
the Users who are the controllers to determine the legal basis of
processing the personal data of Badge Earners. The processing of personal
data is necessary for the Service Provider to perform the Service and the
related promises and obligations based on the contract entered into between
the User/Issuer and the Service Provider. The use of Badge Earners personal
information will be limited to the purpose of providing the Service to the
User/Issuer.

Badge Earners' Consent. When the User/Issuer issues a Badge,
the User/Issuer shares the Badge Earner's personal information with the Service
Provider. It is the User's/Issuer's responsibility to obtain the consent of the
Badge Earners. The User/Issuer represents and warrants that the Badge Earners
have consented to the collection and use of their personal information for this
purpose and disclosure to the Service Provider and understand how their
information will be used by the Service Provider. The Service Provider may
request a copy or evidence of such consent.

When a Badge Earner receives an email notice of having earned a Badge, he/she
may (i) ignore the notice, (ii) download the Badge to their computer, or (iii)
direct the Badge to be transferred to a personal account on CanCred Passport, a
separate free storage and sharing platform provided by the Service Provider for
issued Badges serving Badge Earners. Irrespective of his/her decision, User
will retain a record of the earned Badge containing the Badge Earner's email in
their Service account. Users are responsible for protecting the personal
information of their Badge Earners. For more detail, please refer to the
CanCred Passport Privacy Policy.

COOKIES. The Service Provider will also process personal data
connected to its use of cookies that are strictly necessary to provide the
Service by managing logins and keeping sessions open. No further cookies are
used for any other kind of tracking purposes.

For more information on the use of cookies, please refer to the "Cookies"
section of this Privacy Policy.

Legal Basis in the European Union. The processing of personal
data related to the use of strictly necessary cookies is necessary for the
Service Provider to perform the Service and the related promises and
obligations based on the contract entered into between the User and the Service
Provider. Without use of the strictly necessary cookies, Service Provider is
not able to provide the Service.

DISCLOSURE AND TRANSFER OF PERSONAL INFORMATION

The Service Provider does not sell, trade, or otherwise transfer to outside
parties the User's or Badge Earners' personal information. This does not
include trusted third parties who assist the Service Provider in operating its
Service website, conducting its business, or servicing Users, so long as those
parties agree to protect the personal information with a level of privacy and
security protection which is same to that offered by the Service Provider.
Third parties can only use or disclose personal information for purposes which
have been authorized by the User in this Privacy Policy to provide necessary
services to the Service Provider as specified in the contract between the
Service Provider and the third party. Third parties must return or dispose any
shared personal information upon completion of the contract between the Service
Provider and the third party.

The Service Provider may also release the User's personal information when the
Service Provider believes release is appropriate to comply with applicable
laws, the reasonable requests of law enforcement, enforce Service website or
Service policies, Terms of Use or protect its or other's rights, property, or
safety.

Only the username, which each User uses to define themselves, is displayed to
other users in the Service.

The Service Provider may use the following third-party services, provided by
service providers, who are called subprocessors under GDPR: software
development, platform maintenance and hosting.

For residents of the European Economic Area (EEA) or
Switzerland, please note that the personal data the Service
Provider obtains from or about Users is transferred, processed and stored in
Canada, outside of the EEA or Switzerland for the purposes described in this
Privacy Policy. Canada, for organizations that are subject to PIPEDA, such as
the Service Provider, is considered a jurisdiction that offers an adequate
level of data protection, as required by GDPR.

PURPOSES FOR COLLECTING, USING AND DISCLOSING PERSONAL INFORMATION

The Service Provider collects personal information for the following purposes:

- to open User accounts
- to verify the identity of Users of the Service
- to deliver the Service
- to inform the Users of updates, modifications and other matters relating to the Service
- to issue and display credentials of Badge Earners
- to provide support and customer services to Users
- to process payment, send purchase and billing confirmations and reminders
- to plan and develop the business activity of the Service Provider through various research methods
- to investigate and follow up in cases of suspected misuse of the Service
- to comply with legal requirements

COOKIES AND SIMILAR TECHNOLOGIES

What a Cookie is. The Service website may use "cookies".
Cookies are small text files offered to Users' computers or devices by servers
in order to keep track of a browser as a User navigates the Service website.
Cookies may be stored on a User's hard drive, or in temporary (cache) memory,
in which case they are deleted when the User shuts down his/her browser or
turns off his/her computer or device.

How Users Can Disable and Delete Cookies. The User can disable
cookies using his/her Internet browser's settings. Note that if the User
disables cookies, certain features of the Service website may not function
properly. For more information on managing cookies, please go to www.allaboutcookies.org.

STRICTLY NECESSARY OR FUNCTIONAL COOKIES. The Service Provider
uses only strictly necessary cookies. The most important cookies are the
functional or strictly necessary cookies that are written onto the Users'
computers or mobiles device for browsing, optimizing and customizing purposes.
They are essential and help Users to navigate on the Service website and the
Service and to use basic features. These cookies are strictly necessary to
provide the Service by managing logins and keeping sessions open. No further
cookies are used for any other kind of tracking purposes.

SOURCES OF PERSONAL INFORMATION

The Service Provider receives personal information primarily from the Users, as
the data is entered into the Service by the Users. For the purposes described
in this Privacy Policy, personal information may also be collected and updated
from cookies. Data updating of this kind is performed manually or by automated
means.

LOCATION OF PERSONAL INFORMATION

The Service Provider maintains the Service and stores and processes any
personal information collected through the Service on servers located in
Canada. No personal data is transferred outside Canada by the Service Provider.

PROTECTION AND SECURITY OF PERSONAL INFORMATION

The Service Provider and its third-party service providers use appropriate
technical and organizational measures to protect the security of personal
information provided or generated through or received as a result of the
Service.

The personal information is stored in the Service databases which are secured
with firewalls, passwords, backups, malware scanning and other appropriate
technical and organizational measures. For example, the email addresses of
Badge Earners are encrypted to prevent unauthorized access, according to the
Open Badge standard. The Service databases and the backup copies of them are
maintained in locked and monitored premises and can be accessed only by certain
designated persons, i.e. only those of the Service Provider's employees, who as
a result of their work are entitled to process personal information with
designated access rights (username, password and access level information).
These persons include the Service Provider' customer service personnel, the
technical administrators of the Service and trusted third parties.

Some information may be kept outside the Service databases for the purposes of
invoicing. That information is also subject to appropriate technical and
organizational security measures.

The Service Provider ensures its personnel and third-party service providers
abide by the appropriate confidentiality commitments. The Service Provider will
strive to ensure that no stored personal information (i) disappears, (ii) is
used for wrong purposes or (iii) is accessed or (iv) changed without
authorization.

Users are warned not to disclose their username or password to anyone other
than the Service Provider.

Data Breaches. A personal data breach is a breach of security
leading to the accidental, unlawful or unauthorized destruction, loss,
alteration, disclosure of, or access to personal data. Breaches can happen
when personal information is stolen, lost or mistakenly shared. The Service
Provider and its third-party service providers have procedures in place to deal
with any suspected or actual data security breach, including risk assessment of
any suspected or actual breach and maintaining records of all breaches. The
Service Provider will notify the User without undue delay after becoming aware
of any personal data breach and any applicable regulator of a suspected or
actual data security breach where the Service Provider is legally required to
do so.

Links to Third Party Websites. The Service website may contain
links to other websites that are provided as a convenience only, neither owned,
nor managed by the Service Provider, and which may have different privacy
policies and practices than those of the Service Provider. The Service Provider
has no responsibility for these third-party websites, and User is advised to
review the privacy policies of any third-party websites User chooses to visit.

RETENTION OF PERSONAL INFORMATION

Personal information is retained for as long as is necessary for the Service
Provider to fulfill the purposes which have been identified and consented to by
the User or otherwise required by the law. If the User quits the Service, the
related personal information is deleted. The User may request complete deletion
of their account, which will destroy all personal information stored on the
Service server. The Badge Earners can also delete their personal information on
their own initiative.

USER RIGHTS

Under the PIPEDA, Users have right to

- access their personal information in the custody or control of the Service
Provider and have an account of its collection and use.
- correction to have the User's personal information corrected if it is
incorrect, have incomplete personal information completed or out-of-date
data updated.
- erasure to request deletion of personal information processed by the
Service Provider at any time in a number of situations, except for the
following situation:
- the User has an ongoing matter with the Service
Provider's customer service or technical administrator personnel.
- complain to the Service Provider and the Office of the Privacy
Commissioner of Canada, if the User believes that the Service Provider has
processed the personal information incorrectly.

In addition to the above rights, and if the Badge Earners reside in the United
Kingdom (UK) or a country within the European Union, under GDPR, subject to
certain exceptions, Badge Earners have additional rights, such as the right to
data portability and the right to restrict processing. Service Provider, as a
processor of the Badge Earners personal data will assist the Users, who are
controllers of the Badge Earners' personal data, by appropriate technical and
organizational measures, as technically feasible and applicable, to respond to
Badge Earners' requests for exercising their rights, taking into account the
nature of the processing carried out by the Service Provider.

Users can exercise their right to access, correction and erasure by accessing,
modifying and/or deleting any personal information stored in the Service by
logging into their Service account.

In addition, Users can exercise any of these rights by contacting the Service
Provider using the information provided below in the "Contact Us" section. The
Service Provider will respond to any User request within a reasonable timeframe
in accordance with the applicable law.

CHANGES TO THIS PRIVACY POLICY

The Service Provider reserves the right to make adjustments to this Privacy
Policy at any time and from time to time. The Service Provider suggests that
Users review this Privacy Policy on a regular basis.

CONTACT

If Users have questions regarding this Privacy Policy or Users believes that
the Service Provider has not abided by this Privacy Policy or wish to exercise
any of their privacy rights, Users should contact the Service Provider using
the information provided below:

Learning Agents Inc.
Mailing address: 134 Home Street
Winnipeg, Manitoba
CANADA
Phone number: 1 (204) 219-5933
Email address: info@learningagents.ca

Save and continue later

Copy and save the url below. You can return to this address and continue editing later.

Send url to your email address

Click "Save and continue later" if you wish to save your work.

The application is not sent for assessment until you click "Submit application."