605: Navigating College Applications with AI (Part 1): How High School Teachers and Students Use Tools Like ChatGPT

The College Essay Guy - Episode 605 - Jen Rubin_otter_ai

The College Essay Guy - Episode 605 - Jen Rubin Tue, Apr 22, 2025 8:08AM 49:58

SUMMARY KEYWORDS

AI in education, college admissions, ethical use, AI literacy, generative AI, student voice, AI bias, AI detection, college policies, AI tools, AI in essays, AI in recommendation letters, AI in writing, AI in learning, AI in social media.

SPEAKERS

Speaker 1, Ethan Sawyer, Jen Rubin

E

Ethan Sawyer 00:08

Jennifer, Hi friends and welcome back to the podcast. So my conversation today is with Jennifer Rubin, senior researcher at Foundry 10 and principal investigator of the digital technologies and Education Lab. Jen is someone I admire for her research jobs, which are impressive on their own, but also for her curiosity and for her commitment to equity. Jen is a first generation college grad, and like me, she didn't have a ton of support navigating the college application and admission process. So when chat GPT was released, she wondered, how were students actually using these tools in the college essay and application process? And how do we make sure that AI becomes a tool for equity and not just another advantage for students who already have lots of advantages? These are questions that interest me deeply as well questions that interest me deeply. In this conversation, we get into some of the most interesting takeaways from her research. We talk about why higher income students were more likely to use AI in their applications than lower income students, how educators can increase their students AI literacy, how students can use AI without losing their voice, some resources and tools that Jen recommends for students and counselors navigating this new technology and lots lots more. Jennifer Rubin is a senior researcher at Foundry 10 who investigates how technology shapes youth development, learning and social connections. She earned a BA in Psychology from UC Berkeley before completing a dual PhD in psychology and Gender and Women's Studies at the University of Michigan. Currently, she's exploring how young people navigate digital spaces, strategies for educators to meaningfully integrate emerging technologies and the essential skills needed to foster critical engagement with social media and AI tools. Hope you enjoy. You Jen, Hi, Jen, welcome to the podcast.

J

Jen Rubin 02:07

Hi. Thank you for having me today.

Ethan Sawyer 02:09

E

I'm so glad you're here, and I've enjoyed our conversations leading up to this. Just getting into this a little bit, and I'd love to just have you start off and set context for folks and share just a little bit about the why behind this research study and this work you've been doing.

J

Jen Rubin 02:26

I'm a psychologist, and I have been for about 15 years now, and so much of the work I have done has been inspired by real world experiences or sometimes just thinking about my personal experiences and how that might relate to a research question. I'm a first generation college student, and I went to Berkeley of Michigan without a lot of outside help preparing those applications. I really wrote my college essays on my own, and I didn't have that kind of parental guidance or even access to private coaching that some students rely on. So when I think about AI tools like chatgpt and Claude that have come out recently and have been really widely available. I was really curious as a researcher about how are students actually using them in the college application process? And this question really drove our current research project that we've done in our lab here. We really wanted to understand how students were generally using AI, not just for hype or fear surrounding it, but really the real ways it was shaping their application experience as future college students. Some questions we were thinking about was, were they using it for brainstorming ideas? Were they depending on it to write something for themselves? And how did this affect how admissions officers perceived their work? That last question is something that we're still really curious about and plan to research in an iteration of the study of the future. At the same time, we really wanted to look beyond students, and I was super curious about how teachers were responding to AI in college admissions. The conversation around AI is really evolving quickly. We really want to capture this moment while schools were still trying to figure out what AI's role should be in the college admissions process, as well as how students use it once they're on college campus.

E

Ethan Sawyer 04:06

Beautiful, perfect. So what were some of the interesting takeaways from your research? What'd you learn? We learned

J

Jen Rubin 04:12

a lot, but I want to highlight a few of the major takeaways that I find most interesting. One of the most interesting takeaways from our research was that 30% of students reported using AI tools such as chat GPT to help with their college essays. Despite this, students are really grappling with ethical concerns of using these tools, and we tested this by asking students to think about ethics, how they were using the tools, and any fears they might have with using these tools in the college admissions process, and they had a lot of really complicated feelings around fairness, authenticity, and the extent to which AI Generated Content reflects their true abilities. A particularly interesting finding was that while students and teachers generally agreed on the ethics of students using AI tools in their essays, they. Agreed when it came to teachers using AI for recommendation letters, students saw both cases as ethically similar, while teachers viewed their own AI use as more acceptable than students use in their own essays. For me, this really raises important questions about who gets to use AI and for what purposes and why different forms of assistance might be judged differently by adults and youth. I think finally, one of the most interesting parts of this research for me is that we also ran an experiment. I was really, really interested in people's perceptions of using a human college admissions coach versus something like chat, GPT or generative AI. And what we found was that students and teachers perceive AI assisted work by a college applicant as less authentic, competent and ethical than work done with human help. This is super fascinating, and something that since the publishing of this white paper, we've explored a lot more with many more experiments, and I'm happy to talk about that at a later time, if you want.

E

Ethan Sawyer 05:58

Yeah, I find that really interesting. One of the things that I also found interesting is that in your research, you found that higher income students were actually more likely to use AI in their applications than lower income students. What do you think this is about like? What explains this trend? It's

Jen Rubin 06:15

J

a bit counterintuitive. Right. Going into this research, we have read a lot of opinion pieces in the New York Times and other outlets talking about how generative AI could potentially be a leveling of the playing field for lower income students who might not have access to resources to help with their college admissions essays. But what we found was kind of the opposite. We found that only about 22% of students from families earning $50,000 or less reported using generative AI tools in college admissions. In contrast, 40% of students from families earning between $75,000 and $100,000 engage with AI use in their college admissions process. This is 150% increase in odds of using AI compared to students from families earning $50,000 or less. I do want to caveat that we are exploring this socio economic status income finding and further research, because we did find this difference that was statistically significant, but some of our other income groups didn't have a statistically significant relationship between their use and we really want to tease this apart just to understand the role of income in students use of AI in the college admissions process. Why do you think that is I can think of three things that might be driving this finding. The first is awareness and access. Potentially higher income students may have more exposure to AI. They might have parents that are really tech savvy and use AI in their own work. They might also have access to college counselors or teachers in their schools that encourage the use of AI. Lower income students might not have the same level of digital literacy, or might not even need know these tools exist if they are in schools that might not have access to teachers that teach about generative AI within the school context, I think there's also something about confidence here, right? So there is a learning curve with using generative AI. Perhaps wealthier students who may already feel more secure in the missions process, might be more willing to experiment with it. Lower income students might not have the same interest in experimenting with this tool because they might be more hesitant, or they just don't have that confidence built yet. I also think there's a lot of institutional barriers that we should also be considering, and I think this comes down to which types of schools are both encouraging its use or discouraging its use. Anecdotally, when we've talked to some teachers, we have found that some teachers from our private schools were really on the forefront of using generative AI in schools right when these tools came out. We really learned about this with some of the focus groups we talked to and then some of the teachers that we work with at Foundry 10. In contrast, we found a lot more hesitancy from some of our public school teachers. It might be possible that students in lower income schools might not have the same access and enthusiasm when it comes to generative AI. AI must also be framed as cheating rather than a helpful tool, and if it's framed in this way, you're probably more likely to want to avoid the tool altogether.

E

Ethan Sawyer 09:16

Yeah, as soon as chatgpt came out, there was a lot of folks speaking up and talking about, oh, this could be a tool for access and equity. And there hasn't been a lot of like that. I've seen a lot of practical advice on this, you know, and I'm somebody who thinks about this a lot. And side note, like, I'm teaching a course right now for counselors, and we're talking about AI that was like, the theme of last week, of like, what's the new future look like? But I'd love to hear what are some practical examples when we talk about access and equity that, for instance, educators, counselors or teachers listening to this, how can they put these AI literacy? Let's call it into practice

Jen Rubin 09:54

J

in our lab, we think a lot about AI literacy as well as responsible and ethical use. So this is really at the heart of so much of the research that we do. I have a few ideas. I think, first and foremost, we do see some schools rushing to ban technology. I don't just mean AI. I also think we can think about cell phones and other forms of technology in the classroom. I think for us as a lab, we're much more about teaching literacy and responsible and ethical skills instead of an outright ban. So when we're thinking about generative AI, we can think about this as explicit AI instruction, thinking about how students should incorporate AI or not in their work. And I mean this really thinking about the types of ways that AI should be incorporated. For example, we can think of AI as a useful tool for things like brainstorming, outline and revision, but we also really want to make sure that students work really reflects their true and final voice. So we really want to think about AI as a helpful tool, as opposed to something that does the work for you. It's really important to think about like scaffolding AI use as well. And I mean this because AI can have somewhat of a learning curve. We have to think about really strong prompting, for example, when it comes to using AI, and this is really a skill that can be scaffold and taught. So this approach really involves guiding students to craft specific prompts, critically evaluating AI responses to make sure that they are truthful and based in actual fact, as well as refining their own thinking. Instead of accepting AI outputs at face value, I really want to stress that it's so important to understand that AI can also produce misinformation and hallucinations, meaning that the content that it produces is not factually correct, which is why it's so important to teach these skills within an educational studying with teachers as well as peers. I think it's really important as well to think about ethical discussions. I mentioned some reasons or some ways that we can use AI, for example, for brainstorming. But where does that line really cross and maybe to some more unethical behaviors. So therefore, I think we should be thinking about CO designing ethical guidelines and responsible use in schools, both with counselors and educators, as opposed to just maybe having a top down approach where those rules come from adults, in and of itself. And my final kind of point that I want educators and college counselors to think is about access and transparency. I strongly believe that schools should provide AI resources to students so that it doesn't become another tool from which only certain students benefit. That might mean offering workshops, integrating AI into college counseling, or even ensuring students can access reliable AI tools. I, as a researcher, am not an evangelist for AI, but I am a realist when it comes to the role that AI might have in the future of our lives. I mean this not only in education, but also the workforce. We do know that AI is a tool that has advanced rapidly and what it can and cannot do, and we really should think about it as another tool that might be integrated into the vast amount of the toolkit that we have, as opposed to something that might fizzle out in the future. Yeah,

E

Ethan Sawyer 13:02

I'm really appreciating what you're saying. It's, it's, it's also lining up with research that I've done around what colleges are looking for and and even the common app. So when students submit their application, they have to sign this affirmation statement from the common app that says, I certify that all information submitted in the admission process, including this application, any other supporting materials, is my own work. One of the interesting things that I think you're kind of pointing at here, the sort of gray area, is what my own work, quote, unquote means, and what it represents. And, you know, colleges are coming out just to, sort of like really support what the point that you're making, or to sort of echo it, you know, Caltech, for instance, says, you know, it lists in terms of ethical uses of AI, using, you know, for instance, generating questions or exercises to kick the brainstorming process off, or, you know, researching the college application process, or even using things like checking grammar and spelling and then under unethical uses. And this is on its website, and I'll link to it in the show notes. It talks about, you know, copying and pasting directly from an AI generator and relying on, you know, an AI to just, just straight up write the essay. But it seems like there's some of this, like, in between area where it's like, when we talk about things like, you know, a student voice, it feels like we're in this, this period of like, all learning it together and figuring out what does that even mean. Like, what are do you have thoughts on, like, how students can use AI without, you know, losing that quality of what we call, sometimes authenticity or their voice. But I think

Jen Rubin 14:36

J

this is one of the biggest challenges that we need to think about as technology becomes more advanced. AI is really great at making writing sound polished, but I think it could also make essays feel really generic. And when we're thinking about college admissions and this essay that we're writing to show who we really are to schools, we definitely want something that does not sound generic. We want it to sound authentic and like it's coming from ourselves. So. And in this way, we really need to think about what AI is good and not good for. So I really think that the key to AI is really viewing it as a brainstorming partner and not as a ghost writer. I think that there's three different ways that students can really maintain their voice in the essay writing process if they want to think about exploring generative AI tools as a way to help with that process. So number one, I think it's so important for students to use AI for structure and not substance. Ask AI to help outline an essay or suggest topic ideas, but really making sure that the core ideas come from the student and their own experiences. I think it's also important to first write out a draft that reflects who you are, and then go back and refine, refine later. So this can mean drafting an essay and then using AI to help clarify or improve phrasing. Something that I do, because I write a lot in my job as a researcher is I often will write something and then I ask AI to analyze the writing that I do to make sure that it is clear, that it's understandable, and that I don't have any gaps in my way that I'm setting up an argument. I think that's really important, and we can think about this for student use as well. We can use AI as that brainstorming partner, even that tutor, that can help us identify some challenges in our writing in order to improve it. Third, I think it's really important to use AI to enhance storytelling and definitely not replace it. Instead of letting AI generate full paragraphs, students can use it to ask reflective questions or even help rephrase ideas more clearly. A great personal statement has emotion, has unique experiences and really that strong sense of self. And we can think of AI as helping to polish it, but the student story should really be front and center. So I just want to remind students and caregivers and counselors out there listening that really colleges want to see authenticity and originality should be a reflection of a student's lived experiences, their values and their aspirations and what they look towards towards the future. And I really think it should read like a conversation with the applicant. I know that admission officers are really looking for students to articulate their personal growth and their perspectives, and AI can be a helpful tool, but it definitely should not flatten individuality, because I often think the best essays are the ones that have been written by that student. Yeah,

E

Ethan Sawyer 17:18

and that's something that, again, I'm finding in my research from colleges. I'm just going to quote another one, because it just occurred to me, just as you said it, I'm the Dean of admission at Princeton, Karen Richardson, and this is on their website. Said AI is not inherently bad, but I caution against it in your college application. First and foremost, I guarantee that any essay one writes with the help of AI is not going to be nearly as good or authentic as one that an applicant composes on their own. And remember, we're trying to build a community of learners who will contribute to the academic and social fabric here, so we count on authenticity in the application. So again, sort of echoing what you're saying, like colleges are in line here. But the other thing, and this is something for maybe a future podcast, is like it really gets, sort of my, as my friend would say, squiggly when it comes to like this notion of authorship and and what, one of the things I would encourage counselors to think about is, what are some of the tried and true things that we believe represent authenticity, and what happens if we were to Suppose that a student could have a quote, unquote, authentic experience and also potentially be using AI. Like, is there any room for this? And this may sound to some folks like, what the College Essay Guy is saying this, but I don't know. I'm starting to look at, for example, my own, you know, interactions with AI and having it be a thought partner has been incredibly interesting and useful, and it asks me really hard questions. Like, you know, I'm gonna give a couple of these here, and this is probably the subject for a future podcast. But, you know, things like, Do tools like aI diminish ownership, or do they just change it, change what we mean by ownership? You know, could it be that this because one of the other critiques is, like that it's just speeding things up. And I wonder if speed and depth are actually possible. And so one of the things that I'm wrestling with is thinking about, what are those things that I'm I'm really, like, feel like that. No, this is the thing that has to stay. And we can't do this with AI, like, what really are those things? So I, as folks are thinking through this, Jen, I'd be curious to know, what are some like, resources or tools that you would encourage, that maybe students who are navigating this new technology, you know, could could engage with to start to think through, what does it mean to ethically and effectively, you know, use AI if they're planning on, you know, using it or considering using it, this

Jen Rubin 19:36

J

conversations we should be having with youth throughout school and also at home. I think we can't forget the family piece to this, because these conversations are a way for us to think about responsible and ethical use what that looks like in different contexts. So I have a few resources that I always give to educators and caregivers, if they ask me, I think the first is really having students and families. Look at Common Sense media's AI, literacy resources. Common Sense Media is an excellent research evidence based resource for communities. I think what really makes their resources really strong is that it's often age appropriate, and it really spans guidance for students, caregivers, educators, and thinks about ethical use, critical thinking, transparency, all these things that we've been talking about in our conversation today, in school, work and beyond. Another really big one that I use with families and students is digital promises, AI and ethics initiatives. Digital promise offers research backed resources on AI in education, digital equity and responsible AI use, they have this really great framework when it thinks about ethical AI use that I really encourage everyone to take a look at. It's freely available on their website, and there's a lot more resources based around this framework that they've built over the past couple of months. Another really important thing is to look at university AI guidelines. In my mind, I wish there was just a blanket University AI policy across universities in the US. Unfortunately, we know that that's never gonna happen, and it really is university based, and we also know that a lot of those policies haven't been developed yet at the time of recording this podcast. So I think it's important to look on the College website, understand University AI guidelines. Ask college admissions counselors both at the college in and of itself that you're interested in applying to, but any guidance counselors you might have at your high school to see if there's a way to understand what transparent use looks like when applying to that school, and I mentioned parents and caregivers. My lab, in collaboration with researchers at the University of California Irvine, recently published a white paper addressing caregiver experiences with generative AI, and we really thought through recommendations for facilitating conversations about AI at home. We offer conversation starters, we offer our research, and what we found from talking to focus groups of caregivers nationally. So I think it's a really great resource. It's available on our foundry 10 website, through my lab, the digital technologies and Education Lab, and I think just to touch a little bit more upon responsible and ethical use, I think the two most important considerations include, once again, using AI as a tool and not as a replacement. Remember, we want it to reflect your own voice and your own experiences, as well as to think really critically when you're using AI as a tool. I mentioned that AI is not perfect. It can introduce bias, inaccuracies or even generic phrasing. Students should always review, edit and personalize anything that AI suggests to ensure that their final submission is both authentic and uniquely theirs

E

Ethan Sawyer 22:52

beautiful. So we talked a lot about the student perspective, in particular, as it relates to AI use in say the personal statement or supplemental essays. But I'd love to talk a little bit about teachers use of AI. You know, Do do you see ethical concerns around, for example, AI generated recommendation letters, and you know what are in terms of the what the your studies revealed? What did teachers have to say about their own use? So

Jen Rubin 23:17

J

we did talk with high school teachers that write letters of recommendation for students going to college. What we found was that 31% of teachers that we sampled reported using generative AI to help with their letters of recommendation. This included a variety of things so we just don't want to say like writing the complete letter recommendation. It included things like brainstorming, word phrasing, clarity of argument, as well as a few other things, such as helping save them time. So maybe thinking about editing that letter for them to make sure that it is in the best shape possible before sending it off to a college for that student to be considered for acceptance. I do think that AI generated recommendation letters sit in an ethical gray area. It's not necessarily wrong, but really requiring careful and responsible use. And I think we should have considered this, just like students and the advice that we're giving our youth, teachers should follow the ethical and responsible use guidelines when integrating AI into their work. These can be guidelines that were developed internally at their school, in the classrooms that they're involved in, or within the district that they sit in. One of my biggest concerns with this ethical gray area is personalization. We know what it takes to write a strong recommendation letter. I have written dozens of recommendation letters for students going to graduate school so a slightly different contexts are college graduating seniors, and I know what it takes to write that strong recommendation letter. It's really supposed to reflect the student's unique talents, what they've done in school, their character and their potential. And I think that if AI generates large portions of these letters, does it still carry the weight of a teacher's professional judgment and personal. Experience AI generated, writing risk kind of sounding formulaic, really missing the depth and nuance of what can make a recommendation impactful. Another issue I see here, just with our students, is the issue of transparency. Some questions I have include, should teachers disclose that they are using AI drafting this letter, we have debate over students and disclosure in their personal essays, and I think that same question really applies to teachers. If AI primarily generated a student's letter of recommendation, does that make the endorsement less meaningful? I do think that college and university should provide really clear guidance on what is access acceptable, since we see this uptake in teachers experimenting with AI and this recommendation letter process, I think with that said, I do want to really highlight that AI can be a valuable tool for teachers if used appropriately. I know what it's like to handle dozens of recommendation letters in one season. It takes up so much time on a plate that's often really full for educators. So we really want to think about that time aspect and how educators can use AI to help structure letters, refine phrasing, brainstorm language. I think those are all really responsible ways to use the tool. The ethical approach is to use AI as an assistant and not as a replacement for teachers, and that's the same advice I extend to anyone using these tools. I do just want to remind teachers that just should be reviewing, editing and personalizing every letter to ensure that it truly reflects their relationship with that student.

E

Ethan Sawyer 26:35

You're asking such a great question here. Jen, of like, you know, if the letter represents, you know, the teacher's beliefs, the teacher's thinking, like, does it matter? You know, if it doesn't matter, if it was generated by chat, G, P, T, and, and then there's the larger question, which is, like, and it's going to be, I think, particular to the individual, but it's, it's one that we're wrestling with, is like, well, then what's the difference? Then, if so, it seems like counselors and teachers are more okay with themselves using it, but not okay with students using it. And there's a big gap there. And I don't really know what to make of that. Like that seems to be, right? That seems to be what the what was shown in the study, right?

Jen Rubin 27:15

J

Yes, that was exactly what was found in our study, and it's a statistically significant difference, this gap that we see with teachers perceptions of the ethics of a student using this as a teacher's perceptions of themselves using an AI tool in their letters of recommendation, which is so interesting.

E

Ethan Sawyer 27:31

Yeah. Do you have any sense of, like, what that's about? Yes. So

J

Jen Rubin 27:35

we've thought a lot about this. I've also talked like, once again, with educators and focus group settings with our qualitative research, something that's a big concern for educators, which is a really valid concern, is skill development. Are students losing skill development when we incorporate generative AI tools into the work that they do? Teachers have a slightly different perception of themselves. They are adults. They have gone through college, perhaps graduate work. Yes, they developed lots of skills over the course of their decades of work, and they see it as like a different use of that tool, as opposed to students who are still very much in that learning process and phase of their lives. Yeah,

E

Ethan Sawyer 28:14

yeah. The way that I think of this is like nutrients, like if we just rush to focus on the outcome, aka the college essay, for instance, and we skip, if the student skips over writing, the writing process, I feel like there are all these valuable nutrients that they miss out on. And I've been writing about this for the last few weeks. But you know the sense, for example, the sense of ownership that comes from looking at that piece of writing and being like, Yes, I worked hard on that, and I wrote that, and you know, the emotional intelligence, you know that can, I think, develop over the process of like asking yourself deep questions and thinking critically about your experiences and the meaning making that comes. I'm sort of getting on a soapbox, so I'm gonna pause. But you know, it seems like even Besides that, there are all these other things that potentially students miss out on, but I'd love to hear because I think there are some colleges who are coming forward I mentioned a couple earlier in saying specifically in their guidelines, here's our take, but there's I'm just curious, what have you seen so far in terms of how colleges are approaching AI, use And applications? You know. Are they who's setting clear guidelines? Is it still a gray area for most like, what are you noticing?

Jen Rubin 29:25

J

Overall? I do believe that college and universities are really still developing those policies, and I've really only seen instances of a few universities having a very formalized policy. So for example, I know that Yale informally discourages AI use, really emphasizing authenticity on their website. If you would go search for this, the University of California has a much more extensive or clear AI policy. They allow AI for brainstorming, for example, but really expect that final submission to reflect that student's own work. I've also seen policies from Harvard and Columbia. Yeah, they really focus on ethical AI use. So that means like encouraging experimentation while warning about privacy risks, for example. And Caltech kind of goes a step further from what I've seen, requiring applicants to review AI ethics guidelines before submitting their essays, so really making sure that students understand what ethics looks like at Cal Tech before engaging in that writing process. I think it's immensely important for colleges and universities to take this seriously and develop really clear policies that students have when going into the college admission season. It's really hard to be in this gray area if students don't really have a clear ethical or responsible use training in high school, and then are entering this phase where that might not be really, really clear when they're submitting their essays, or even thinking about entering a university or college in and of itself, I really think, oh, sorry, go ahead. No,

E

Ethan Sawyer 30:53

no, please. You go. You go. If

J

Jen Rubin 30:54

students use AI, I think they should do it thoughtfully by making sure their application still reflects their voice, ideas and experiences. And I think that colleges and universities play a key part in really showing students what that can look like.

E

Ethan Sawyer 31:06

Yeah, I just want to, because we are sort of in this liminal time right now where there isn't specific guidelines from, you know, for example, every institution, like you said. And I just want to, like, speak to students and say, like, if y'all are considering using AI, first of all, I just want to kind of encourage you all to be conservative. In other words, I want you to be able to specifically say, here's how I use AI. And if there are situations where there are cool ideas coming out of the machine, you're like, oh, that sounds interesting. Like, yes, please make it. Make it your own. And I just want to, like, encourage caution for students. I'm going to have more specifics that are going to that I'll be releasing on the blog at some point soon. But for now and then, the absence of sort of like, what do the schools say? You know, the sort of authority figure? Yeah, because I have seen instances where, and I've heard about instances where students, you know, have had their applications rescinded, you know, revoked from, you know, because there was so much AI in their application. So I just, I want students to be able to like, to not to have that not happen, in short. And I just want to encourage caution. And, as you said, so beautifully, you know, thoughtfulness.

Jen Rubin 32:16

J

Can I add? Can I add one more thing before going in? Yeah, please. I do want to mention to colleges and universities that AI detection software does not work. We hear this time and time again when talking with educators that they wish there was a software that can detect AI or they themselves are using those commercially available AI detectors. But I do really want to caution that those AI detectors often have false positives when we know that the student actually wrote that work themselves, but it's coming back as AI generated. I've seen a couple of really cool examples about how this has been shown. So for example, some folks I know have put in old journal articles that they wrote well before the dawn of generative AI being commercially available, and they'll receive back that this journal article was 70% written by generative AI, which is impossible. So I just really want to caution that and let everyone know that we should be wary about using these those tools, because they could have really disastrous consequences for students. Yeah,

E

Ethan Sawyer 33:13

so here's a question that came up on a webinar just last week, and I'd love to ask it of you, what does a student do in a situation like that? Let's say they've written the essay. It's their own work. They know it's their own work. They put it into an AI detector, and it comes back 70% AI generated. Like, what advice would you give a student in that situation?

J

Jen Rubin 33:34

I think that's a really tricky and important question. I know in the college level, there have been some lawsuits recently challenging that something's been AI written, but I think from a more practical level, if you're writing your draft in Google Docs, they do a really great job of tracking your revisions as well as the kind of arc of your essay, right? So having that like time stamped work shown, I think can be really helpful when we live in an age where perhaps AI detectors are still being used in ways that might be used in like disciplinary action with students. I think another thing is really to have clear education for educators and college admissions counselors to know that these tools are not accurate. I think it's a really scary time when so much is on the line for students and they might be accused of using AI. I think the only way that I see this as being slightly different is that there have been cases when students have left in things like generated by chatgpt, or the model that it's generated by, which clearly is a violation and very obvious that you have used AI or sometimes teachers, you know, you might have read this in pop press articles, have put the instructions and some white text in the instructions. So when you actually caught a cup and you paste that into, let's say, chat GPT, it might produce that thing in the white text that you weren't supposed to do. But I think it's really being clear that these tools. Work, seek out the help of a caregiver or parent if this does come up and you genuinely did not use generative AI to write this essay or do this work that perhaps is coming up as AI generated, yeah,

Ethan Sawyer 35:12

E

that's such a great answer. It's like, save your work. You know the, you know, the old school copyright way. I'm not suggesting that students do this is like, if you had something that you wanted to prove that was yours, you would mail it to yourself, you know. And like, it would be like, you know, have stamps and stuff on it. Be like, this is truly my work. But I think one, the Google Docs idea is a great idea. Shout out to Google Docs. And then the second thing you've kind of alluded to this, but like, having somebody who you know, your counselor, for instance, your college counselor who has seen drafts and has someone who's sending of your writing. So if you do get in trouble, you've got somebody who can vouch for you. Can be like, no this student, I saw several drafts, and you can even, you know, say, can I email these to you now? For students listening to this and they've already written their essays, the ship might have already sailed. But for those of you who are getting ready to go through this process, like having somebody who's been following along with you, I think can be really helpful as well. So I just want to put in word for that. So my sense, just based on what I've seen, Jen, is that your organization, foundry 10, was some of the first folks to put out research on this. Like, I didn't see too many white papers before this on the role of generative AI tools in the college admission process. But I'd be curious to know, since you released your work, what else have you seen out in the field on this particular topic?

J

Jen Rubin 36:29

I haven't seen a lot yet. But also, if you're thinking about the journal publication process, that often takes a really long time. So this work might have been done, it might be under review and just not published yet. So I'm very eager to continue looking at the research, as well as looking at gray literature, such as white papers, to see what has or will come out in the future. I do want to note that we're actually continuing this work. We plan to launch another iteration of this study for the next cycle of college admissions. We're really excited to see what has changed and what has not changed since we first launched the study for the last college admission season. So we'll have some comparison data, but it won't be for a while, unfortunately. So do stay tuned. Got it, maybe we can talk about it again in the future. Totally,

E

Ethan Sawyer 37:16

we'll do a follow up in six months or a year. So let's get personal for a second in terms of our own AI use. I'd love to hear and I'm happy to share as well, like, how do you use AI if you do in terms of your work and your writing?

Jen Rubin 37:29

J

I definitely use generative AI in my work. When chatgpt became commercially available in November of 2022 as someone that has formerly worked in tech and someone who studies Tech, I was super excited to see what this tool can do. So I was definitely an early adopter in exploring the power or the also the boundaries of chat GPT. And I will say that the technology has improved a lot over time, just in this short amount of few years since it's come out. And as a researcher, I mentioned a little bit earlier that part of what I do is writing. And I write so much, I write reports, I write white papers, presentations, journal articles, for peer review publications. And I really do think that AI can be a great tool for getting quick feedback on clarity, logic and structure. Part of what I have to do is write for a variety of different audiences. So I could be writing for youth that are in high school, for example, for teachers, for academic researchers, and each of those audiences requires a different level of how I present my ideas. So for my academic researchers, I'm going to use terms that are common in the field and the area that I'm working in, but I don't want to use those complex terms necessarily when I'm talking to a group of high school students or even middle school students. So I do have aI help me refine some of my language to be more accessible so that I can really ensure that younger audiences can engage with the material at the level that they are at. So for example, I might use AI to help rephrase a sentence for better readability, suggest alternative wording, or even identify areas where my argument could really be stronger. It's especially useful when I've been working on something for a long time and really need a fresh perspective. I've been working on a new white paper about social media literacy curriculum, and we've been thinking about this for a really long time, and I was really stuck with the structure of the paper, making sure that it really flows nicely, that there's transitions between the sections and our findings, and then it really creates a cohesive story. So I definitely use generative AI to help give me critical feedback on my writing so I can go back and make those edits that I need to do on my own words in order to make that paper just stronger and more cohesive overall. Another thing I do is I run experiments which are super fun as a psychologist, but we also have to create materials for those experiments that are similar across the different conditions that we have with slight changes in wording, for example. And I often have aI help look at those materials to make sure that they make sense and that they're really falling the best. Conventions and experimental research. Now, clearly, as someone that's trained as an experimentalist, I have so much knowledge and background in this, but it definitely helps to refine some of the language I might have in a prompt, just to make it really clear for a participant. So once again, I really see AI as a tool for refinement and not creation, and it really helps me with efficiency, critical thinking and decision making, but all that work still really comes from me. It might give me ideas about how to better those areas, but I'm still doing all the hard work and the critical thinking and the thought process behind the research that we do. Yeah,

E

Ethan Sawyer 40:33

yeah, beautiful. I love that, and I want to share a few more from from my own. But we're, I think we're, we're sort of on the same page, at least, you know, conceptually, one of the things that I've been using it for lately, things that take me a really long time to do, is create presentations like create, you know, slide decks and stuff. And so internally, we found there's some incredible software out there, and I'm not going to speak to the software now, because it's going to be dated soon, and we're going to be using something else. But there's software that we use for creating presentations that I can take, for instance, a huge blog post that we spent a lot of time on and plug it in and be like, turn this into a PowerPoint, a slide deck, and magically will do that. And is that writing? I mean, it's synthesizing, and in some cases, there are words that are sort of like capturing the essence of it, but it's coming from stuff that I've worked on, or that my team has worked on. There's also, this is a fun one, but like, when I have an idea for a particular thing, and it might be like half baked and I haven't really thought through the idea all the way, I might throw it in to chat GPT and say, Hey, what would this look like? Write a business plan for this idea, or imagine this, you know, mapping out over the course of five years. Or what would this look like as a curriculum? And that's been really fun to look at the way that it can really quickly help me think through a vision, you know, of what would a thing look like, and then sometimes internally with the team, I'll say, Hey, here's chat GPT version of what this thing could look like. And so I'm careful to to name it. And then another one is, and this is more specific to like, writing stuff. It could be that, you know, one thing that I've got I spend some time doing is YouTube scripting and creating, turning blogs, for instance, into YouTube scripts. But I can take a blog that I've written on, for example, how to up level your activities list and put that into chat, G, P, T, and say, Hey, write this as a college, as a guy style YouTube video, and it'll and for me, I don't actually go off script. I kind of think of it as like ideas, and I kind of go off as bullet points. But what I'll do is I'll have that pulled up when I'm, like, recording a video, and I'm ethanifying it, which is what we call it internally, of like, you know, putting it in my own words. And so there is, like, a, I guess, a double filter. The first the filter of, like, the content originally, and then the filter of, like, how would I actually say this? So it is kind of still going through me as, kind of as you're saying, but I, but I do sometimes like it for sort of scoping out the big ideas. It can be really fun.

Jen Rubin 42:58

J

J

Jen Rubin 42:58

agree it can be super fun. I think big picture thinking sometimes can be a really big challenge, so having someone, a tool like generative AI, that you can talk back and forth with, can help really refine those thoughts.

E

Ethan Sawyer 43:11

So let's go crystal ball a moment here for a second. Where do you see this headed Jen, in the next five to 10 years?

J

Jen Rubin 43:18

This is a hard question, because the technology is changing so rapidly. You just mentioned that some of those tools that you might mention might be obsolete. I 100% agree with you. I do think that AI will become more commonplace. I do think that when we think about things like Grammarly or spell check, it might be on the same level of that just seen as integration into some of the tools that we use every day, like Google, once again, it's a tool that might just become part of the writing process, but something that's not going to replace the writing process. I have a couple of really big questions that I think about, especially when it comes to college admissions. My first question is, Will colleges formally allow or even encourage AI use in the future? Will there be standardized ways to disclose AI use in a college admissions context or within school more generally, and how can we make AI an equalizer, not just another tool for certain students to use who might have access and training in that tool? Those are the questions that I'm grappling with as a researcher, and I don't really know what the future is going to hold. I'm excited to see where it's going to lead us, and I will be here to research some of those effects and some of those thought processes behind what we're going to be seeing in education.

E

Ethan Sawyer 44:32

Your questions are my question, so I'm feeling really, just really resonant with what you're saying here. You know, it's, it's, it's, this is kind of, I think this is a hard question, but I usually like to give folks some sort of next steps. So as we're sort of thinking for and let's think about different audiences, like, maybe, if you're down to do it, I'd love for you to give some sort of like, so what? So what do we do about this? What do students do? What do parents caregivers? What can they do? And then maybe, last maybe you could share a little bit about counselors, as we think about sort of like, how we navigate this together.

Jen Rubin 45:07

J

Yes, I think that the biggest takeaway I want to leave your listeners is that I really do believe that AI is here to stay and will have an impact on the college admissions process. And once you set like and like you mentioned, I think that students, caregivers and counselors should really be thinking about this in a thoughtful and ethical way. So first students, I think my two things I want to leave you is that, first, to learn how to use AI effectively, experiment with AI for brainstorming and refining ideas. But once again, don't rely on it to write essays for you, your voice and experiences are really what matters most, and I think that's so important for it to shine through in your writing and how you're presenting yourself to colleges and universities. Second, I really want you to chat college policies, as we mentioned a little earlier, schools have so many different roles on AI use, and some might not have any at all right now, but hopefully will in the future. So really make sure that you understand what's acceptable for each school that you apply to, and adjust how you are submitting or creating your materials accordingly. For our caregivers, you're really an important part in this conversation, because these conversations with your children also need to happen at home. I really think that caregivers could encourage critical thinking, thinking about how AI can be a great tool, but students should really still develop their own ideas and writing skills. So really focusing on that skill development that is so important to students learning, it's important to help your children see AI as a support and not a shortcut, and really talking with children at home about responsible AI use and skill development, I think it's really also important for caregivers to be informed colleges are still shaping their AI policies, so really keeping an eye on updates from schools that your student is interested in, and really helping them understand those policies so they can make the best application they possibly can for universities and for our counselors. I think it really comes down to education. So first is really teaching responsible AI use. It's important to help students understand when and how to use AI while still maintaining their voice. You can offer guidance on using AI for structure and feedback, and really kind of show the ethical gray area and lines when AI crosses into maybe a not responsible use case. And the last thing I really want to leave you with for counselors is two other points. So the set, the second point I want to talk about is really raising awareness about AI bias. I touched on this in the beginning, but our experimental findings showed that students who used AI in their essays were perceived as less competent, ethical and authentic than those who received human coaching even when the writing was identical. This highlights an important bias in how AI assisted work is judged. Counselors should help students and educators understand these perceptions and how disclosure of AI use might impact their applications. I also want to mention here with this raising AI bias and awareness is that if we really are encouraging students to cite their use of AI potentially in a college admissions essay, we should really make that clear for all forms of help, and that includes parents and paid college admissions tutors, especially given this bias that we found, I think we just need to be really clear about the types of help students receive and how that plays into the college admissions process. And my final point I want to leave for counselors is really to advocate for clear policies. If schools aren't providing guidance on a AI use, start the conversation. Colleges need to address AI bias and also to ensure their policies don't unfairly penalize students who use AI tools in responsible and ethical ways students should know what's allowed, what's discouraged, and how their AI use might be perceived within the college admissions context.

E

Ethan Sawyer 48:50

Jen, thank you so much for all the work you're doing on this. I'm excited to see what what I'm like, you know, put me in the front row for like, what you release next. And I hope as this thing, as these things shift, that you'll you'll come back and share what you're learning.

Jen Rubin 49:05

J

Thank you so much for having me. This was such a fun conversation to have, and I encourage everyone who's listening to come check out our research, because we have a lot more exciting and interesting work within the realm of generative AI,

S

Speaker 1 49:18

awesome. We will link to that in the show notes. And thanks again. Thank you.

Ethan Sawyer 49:27

E

Thanks, friends as ever for listening. You can catch the show notes at college. Sa guy.com/podcast want to stay connected. Check out our YouTube channel, where we put out a video almost every week when we can and you can opt into anything on the website to get a heads up on our upcoming webinars, courses and tons and tons of free resources on pretty much every aspect of the college admissions process. Be well and stay curious. You.