Over the past year, “AI (Artificial Intelligence)” has been taking over the software world. By that, I don’t mean that AI is writing all of the software, or that AI is taking over the world, but it definitely is the hottest marketing term out there. Virtually everything is getting AI “integrated” into it, and new software comes out every day that uses some form of machine learning algorithm to replace older methods of working with data. Microsoft has been on the leading edge of integrating Copilot—its brand name for any AI-involved product—into the Office suite and Microsoft 365. In this week’s blog post, we’ll give you a short tour of some of its powerful features, as well as some of its missteps, to help you figure out if the time is right to jump on to the AI train.
AI Products at a glance
If you’re thinking to yourself “another post about AI,” I hear you—loud and clear—but the push to get more generative features and better search and summarizing capability into the software that we use every day is only going to increase. In fact, Microsoft was pretty early to the game, starting with Bing Chat (Copilot’s former name) being rolled out to introduce AI web results. From the start, Copilot has been a way to familiarize users with using more natural language to get tasks done on the computer. If you go over to Bing right now, you can simply talk to the search engine as if it is a person, albeit an unreliable one.
Copilot is based on OpenAI’s GPT-4 model, so it has the same quality that you could expect from ChatGPT’s paid features. The different models and products, though, are quite hard to keep straight from one another. On one hand, ChatGPT could give you a little more functionality—like more tweakability—but Copilot for M365 integrates seamlessly with a lot of your existing Microsoft productivity software, making it more powerful for organization and data analysis.
From there, Microsoft’s current and future AI offerings get quite complicated, with certain features being rolled into various products (some of which have quite hefty price tags on them). Copilot Studio, for example, gives you the ability to create AI chatbots that can act as a frontline salesperson or support agent on your website. While that sounds great if it performs its role well, the $200/month price tag might be prohibitive for small and medium businesses. There are more Microsoft AI products coming into the enterprise space, but today we’ll focus on the version of Copilot that runs inside of your Microsoft 365.
Copilot for Microsoft 365
This week, Microsoft made Copilot for Microsoft 365 available to users in organizations of all sizes. At a manageable $30/user/month, what kind of productivity gains can you expect from it and what is it good at? It’s not quite like the enterprise-oriented products that are about developing and deploying AI to your customers; instead, it’s the product that gives you a more centralized and conversational interface for your files, Office apps, and emails. Once you have a license, your Office apps on the desktop and in the browser get Copilot features.
You’ve likely already noticed Windows Copilot, which you can find on your taskbar. That one can do search and change settings for you; ask it to “change to dark mode” or “turn on do not disturb.” The one in M365 is similar in scope, but for your documents and spreadsheets—you can ask it about your documents, use it to draft or edit with you, summarize emails and OneNote notebooks, and perform data analysis in your spreadsheets, and catch you up on what you’ve missed in Teams (in both chats and in meetings!).
Hit: Search and summarize email from Copilot
One particularly useful way of evoking it is to ask it to go through your emails. From your Office.com portal, simply go to Copilot and ask it to summarize your unread emails. What comes back may not be complete, but it can tell you briefly about the topics in the emails and gives you a handy list with links to go look at them. This could be an invaluable feature if you find yourself leaving emails unread to return to them, among one or two hundred that have been marked as read.
Miss: Use summarize to catch up on a complex email thread
When you’re in Outlook, you can find Copilot’s summarize feature just about the reply buttons:
But if I were to take a pretty simple email chain and show you how it summarizes it…
…you might be able to tell that there are some issues with how it tells the story. It told the story in a non-linear fashion, confusing the beginning and ending of the issue. When I first tried the feature, I went and found the most complex email chain I could think of, and it took months of back-and-forth from three organizations and summarized it in two sentences, the second of which was something like “there was a delay in the middle, but the project is now complete.”
In future updates to the LLM (large language models), this might get better, but for now, it’s important to keep the limitations on its summarization abilities in mind. These limitations are not a dealbreaker if you’re using it to organize and make lists (like in the above section); just remember that its job is to search and report, and the more you align your expectations with the results, the better the experience.
Hit: Catch up on Teams chats
Keeping that in mind, catching up on Teams chats, or asking for a summary of what happened yesterday in your channel, is an excellent tool for keeping track of unruly conversations and data. Again, these aren’t going to be complete, or even useful all the time, but if you use it as a memory aid, it will simply remind you of the conversations that you’ve had so that you can follow up on them. It’s helpful that it can give you specifics, but if you already had the conversation, you know the details better than an AI model could. If recording and transcription are enabled in Teams meetings, then Copilot can catch you up, and you can even ask it questions about the meeting as you go. For this feature to work, the host needs to enable transcription, or the option to transcribe when not recording, in their Teams settings.
Miss: Use Copilot to create blog posts (like this one)
Understanding limits is also key to employing Copilot as a writer, but it’s a different kind of limit than the earlier one. Having been the teacher of a writing-intensive course at a university during the release of the first ChatGPT, I very quickly found out what the differences were between human- and AI-generated informative or argumentative writing. Further, being a professional writer at the time made me apprehensive about the public, media, and professional community’s reaction to ChatGPT: “it will replace writing jobs.” Those attitudes are harmful to writers and readers alike, pretending that just because there are words that make grammatical sense on the page that the output of AI writers is adequately engaging, insightful, or well-written.
Copilot is probably most ethically used as a generator for internal communications but not appropriate for external use. Using it to generate all the copy for your website will be detected by Google, for instance, as AI generated content. That doesn’t necessarily mean that it’s going to harm your site’s search ranking, but it does leave that decision up to aggregators and indexers while creating average to over-the-top content.
I wouldn’t have done my due diligence if I didn’t try it, though. When you open Word with Copilot enabled, you get a space to enter your prompt for Copilot to write your document. I gave it some points that I’d like to make and it turned 2000 characters into 800 words in just a few moments. I even referenced some previous writing of mine stored in SharePoint—simply by typing “/” and the first few words of the document—to show it what style to write with.
What it wrote was fine, but it wasn’t exciting or interesting, and (like with the summaries) it had no sense of linear organization of information. The real kicker was the ending, where it said to “keep browsing this blog or watch this video to learn more.” The video that it suggested didn’t exist; it was just something that it plagiarized from inside the GPT-4 model about how blog posts and webpages typically end. Any use of it to write should be edited for accuracy, meaning any long form writing should still be undertaken by a skilled writer.
Hit and Miss—Copilot in Excel
I was excited to see if Copilot in Excel could actually transform and crunch numbers in my spreadsheets, but it doesn’t seem like that’s what it’s intended to do. When asking it to do a routine thing that’s a part of one of my reports, it missed the point entirely: I asked it to round all values in the table up to the nearest 0.25, with a minimum value of 0.5. Instead, it highlighted values under 0.5 in yellow to guide me to them.
Asking it to do the statistical analysis for me doesn’t really get anywhere either:
Instead, it can be used as a conversational search engine for the documentation, which is slick. It’s not as directly involved in data manipulation as I would expect, but that’s probably a good thing for now.
Tip: Use Copilot positively instead of negatively
Fundamentally, some of Copilot’s limitations come from asking it to look for things in a void. The more specific you can be, the better the result but some things are just going to stay strangely out of reach. Asking for a specific email with a specific attachment from a specific person didn’t work for me, and neither did asking for contact information from someone in our organization. I’m not sure why these searches didn’t work, but maybe they will in the future.
For now, it’s safe to say that there’s some utility here in the implementations of Copilot, and hopefully it can incrementally improve through more use and development. If these experiences seem like they’re a little more on the negative side, then forgive me for being a little too swayed by the marketing materials and promise of revolutionary technology in AI products. I can easily see a future where these tools are ready to be a little more in the driver’s seat. Until then, I think the user that can find the most utility in Copilot for M365 is a user that feels like their Office experience is a deluge of information that can’t adequately be kept up with without gaining every advantage possible.
-Written by Derek Jeppsen on Behalf of Sean Goss and Crown Computers Team