Unit 14: What is Generative AI?

Before discussing genAI, we should consider it in the context of the long history of AI.

What is Artificial Intelligence (AI)?

Artificial intelligence (AI) is the theory and development of computer systems capable of performing tasks that normally require human intelligence, such as interpreting language, recognizing patterns from large amounts of data, and making decisions. Some of the names given to AI, based on the way in which it is designed and what it can do, include neural networks, natural language processing, computer vision, speech recognition, machine learning, and deep learning.

Take a look at the timeline in Figure 14.1 to explore a selection of the key advances in AI.

While we interact with AI everyday, usually it hides in the background of the technologies we use. GenAI is different in that we can use our voice and text to command the technology to produce output that we would normally have to create ourselves.

Miao and Holmes (2023) define genAI as follows:

Artificial Intelligence (AI) technology that automatically generates content in response to prompts written in natural language conversational interfaces. Rather than simply curating existing webpages, by drawing on existing content, genAI actually produces new content. The content can appear in formats that comprise all symbolic representations of human thinking: texts written in natural language, images (including photographs to digital paintings and cartoons), videos, music and software code. GenAI is trained using data collected from webpages, social media conversations and other online media. It generates its content by statistically analyzing the distributions of words, pixels or other elements in the data that it has ingested and identifying and repeating common patterns (for example, which words typically follow which other words).

While OpenAI’s ChatGPT was the first large language model to be popularized, other notable text and code generating models, known as frontier models, have been released, namely, BingChat (a version of ChatGPT that has been incorporated into Microsoft’s Copilot), Google’s Gemini (formerly Bard), and Anthropic’s Claude, along with open source models like HuggingChat and Mistral. At this writing, ChatGPT 4 is considered the gold standard, with both Gemini (select versions) and Claude achieving GPT 4-level performance in 2024 (Mollick, 2024).  In addition, image-generating models, such as DALL-E, Stable Diffusion, Stable Cascade, and Midjourney now offer text-based command modes for creating custom images.  OpenAI has also released Sora, which can create realistic videos from text commands. Each model and application displays differing strengths and capabilities as noted by end-user testing (see the work of Ethan Mollick in particular). In addition, given the rapid developments occurring in the field of genAI, the models and applications are evolving equally rapidly, while new ones are being released.

OpenAI’s ChatGPT draws from a genAI model that was trained on a large number of datasets and documents available on the internet to identify and predict how language works (how to create sentences). The GPT in ChatGPT stands for “Generative Pre-trained Transformer”, referring to a type of machine learning involving a neural network  in which a computer learns to perform a task by analyzing training examples. For example, the training data used by ChatGPT 3.5 is a dataset that includes 570GB of data from sources like books, Wikipedia, articles, and other pieces of writing on the internet up to 2021 (Gupta, 2023). The GPT model learned the probability that one word should follow another, based on context or conversation topic. This ability to predict sentence patterns enables it to generate outputs that seem surprisingly “human” – conversational and coherent.

Users engage with the genAI in the form of a written or verbal conversation often referred to as prompts. The GPT acquired the conversational ability through a process of “Reinforcement Learning with Human Feedback (RLHF) – a method that uses human demonstrations and preference comparisons to guide the model toward desired behaviour” (OpenAI). Essentially, RLHF involves using human feedback to train the model in giving outputs that are socially acceptable—imperfect though they may sometimes be.

Fundamentally, tools like Copilot and ChatGPT are designed to employ your prompts to generate a response based on context and conversational topic. You would ask questions (prompts) as well as follow-up questions to fine-tune the GenAI’s responses until you obtain output or content that meets your needs. Even though ChatGPT “sounds” human, it is not. It is not sentient; it does not have consciousness. Rather, it predicts the likelihood that the next word in a sentence should follow the last word it picked. The AI tool has no concept of fact or accuracy. The model simply looks for patterns in the training data, then produces the next most likely word or phrase. As such, human vigilance and oversight are necessary when using the content produced by any genAI model.

Interestingly, giving the same prompt to the same genAI tool does not always produce the same result. Each time a tool like Copilot or ChatGPT is prompted, it often creates a new output. While there will probably be similarities between outputs from the same prompt on the same topic, they will not be identical. This is even more apparent when you use the same prompt across multiple AI products.

Seneca Libraries offers an overview of genAI that covers a description, uses, basic prompting, and citation practices: Generative Artificial Intelligence?

Also take a look at this video, Introduction to AI for Teachers and Students, by Ethan Mollick and Lilach Mollick of the Wharton School at the University of Pennsylvania. These pioneers in the use of genAI make the topic easy to understand.

Attribution

This page has been edited and remixed from the following sources. Supplemental information has been provided by Robin L. Potter.

Center for Faculty Development and Teaching Innovation.  (2023) What is Generative AI? Centennial College. What is GenAI? – Generative Artificial Intelligence in Teaching and Learning (pressbooks.pub)   CC by 4.0.

Coley, M., Snay, P., Bandy, J., Bradley, J., Molvig, O. (2023). Teaching in the Age of AI. Vanderbilt University Center for Teaching. https://cft.vanderbilt.edu/guides-sub-pages

Elkhoury, E. and Prud’homme-Généreux, A. Future Facing Assessments by  is licensed under CC BY 4.0

Hu, K. (2023). ChatGPT sets record for fastest-growing user base – analyst note | Reuters

Paul R MacPherson Institute for Leadership, Innovation and Excellence in Teaching. (2023). Generative Artificial Intelligence in Teaching and Learning at McMaster University  McMaster University. CC by 4.0.

Miao, F. and Holmes, W. (2023).  5.2 A ‘human-centred and pedagogically appropriate interaction’ approach. Guidance for generative AI in education and research. UNESCO. unesdoc.unesco.org/ark:/48223/pf0000386693/PDF/386693eng.pdf.multi CC by 3.0.

OpenAI. (2024). How does ChatGPT work? What is ChatGPT? | OpenAI Help Center

The University of Queensland Brain Institute. (n.d.). The history of artificial intelligence. Image. The University of Queensland. History of Artificial Intelligence – Queensland Brain Institute – University of Queensland (uq.edu.au)

References

Appel, G., Neelbauer, J., and Shweidel, D. A. (2023).  Generative AI Has an Intellectual Property Problem (hbr.org)

Center for Faculty Development and Teaching Innovation. (2023) Generative Artificial Intelligence in Teaching and Learning – Simple Book Publishing (pressbooks.pub) Center for Faculty Development and Teaching Innovation.  Centennial College. CC by 4.0.

Future of Life Institute. (2023). Pause Giant AI Experiments: An Open Letter – Future of Life Institute

Gupta, A. (2023, April). What is ChatGPT and how was it trained? – Paperpal Blog | Paperpal

Mollick, E. and Mollick, L. (2023). Practical AI for instructors and students Part 1: Introduction to AI for teachers and students. https://youtu.be/t9gmyvf7JYo?si=wWzwn8DsSd1FUxWN

Mollick, E. (2024). Which AI should I use? Superpowers and the State of Play (oneusefulthing.org)

Moran, C. (2023). ChatGPT is making up fake Guardian articles. Here’s how we’re responding | Chris Moran | The Guardian

OpenAI. (2023, March 14).  GPT-4 (openai.com)

OpenAI. Privacy policy (openai.com)

Perrigo, B. (2023). OpenAI Used Kenyan Workers on Less Than $2 Per Hour: Exclusive | TIME

Piper, K. (2023). A guide to why advanced AI could destroy the world – Vox

Sampson, P. (2023). On Advancing Global AI Governance – Centre for International Governance Innovation (cigionline.org)

Seneca Libraries. (2023).  Home – Generative Artificial Intelligence – LibGuides at Seneca Libraries (senecapolytechnic.ca)

Shine, I. and Whiting, K. (2023, May 4). The jobs most likely to be lost and created because of AI | World Economic Forum (weforum.org)

Stokel-Walker, C. (2023). The Generative AI Race Has a Dirty Secret | WIRED UK