Mendes gushed she was 'so proud of my man' as she recalled the 'Not My Ken' mockery he endured in the lead-up to the release of Barbie

Eva Mendes defends partner Ryan Gosling over Academy Award nom for Barbie - as she praises him for taking his 'now iconic character Ken all the way to the Oscars' | Daily Mail Online

In snubbing the film’s central voices—Greta Gerwig and Margot Robbie—the Academy continues its tense relationship with blockbusters.

The Oscars Are Confused About “Barbie” | The New Yorker

Some 1,400 are set to be drafted in March to combat units months before their pre-army programs are completed

IDF to recruit soldiers early from pre-military academy and service tracks | The Times of Israel

This first-hand opportunity will allow community members to gain a deeper understanding of the sheriff's office.

Cole County Sheriff's Office to hold Citizen's Instructional Academy this spring | Mid-Missouri News | komu.com

The Baltimore City Board of School Commissioners voted Tuesday to end a virtual online program’s services for elementary students and to close Bluford Drew Jemison STEM Academy West.

Baltimore’s Charm City Virtual to stop serving elementary students; Bluford Drew Jemison STEM Academy West to close

LEONARDTOWN, Md. – Parents and students at Leonard Hall Junior Naval Academy (LHJNA) continue to express their disappointment and concern over the sudden closure of the private school. As The BayNet first reported last Friday, the school’s Board of Trustees decided to close the doors of ...

St. Mary’s County Attorney Addresses Commissioners About Leonard Hall Junior Naval Academy - The BayNet

BOSTON (WHDH) - The announcement of this year’s Academy Award nominees brought with it big accolades for Boston on Tuesday as two movies with localRead More

2 movies filmed in Massachusetts earn Academy Award Best Picture nominations - Boston News, Weather, Sports | WHDH 7News

With many shocked by the Academy's failure to recognise the work of Greta Gerwig and Margot Robbie at yesterday's Oscar nominations – what was behind it all?

Oscars 2024: The real reason Barbie's Greta Gerwig and Margot Robbie were snubbed by the Oscars

Latest Market News · Editor’s Note: A version of this article first appeared in the “Reliable Sources” newsletter. Sign up for the daily digest chronicling the evolving media landscape here

‘Barbie’ stars Ryan Gosling and America Ferrera don’t hold back after the Academy snubs Greta Gerwig and Margot Robbie | CNN Business

The Oscar nominations 2024 are in as Zazie Beetz and Jack Quaid announced the nominations for the 96th Academy Awards this morning.

Oscar Nominations 2024 List: Nominees by Category - Oscars 2024 News | 96th Academy Awards

Academy Xi

academyxi

This article is Part 1 of the ongoing series From Zero to Product Manager. A set of articles that help you transition to a Product Management role. In this post I explain what a Product Manager (PM) is and how the role can differ across companies, I describe the best qualities of an excellent PM. You are aspiring to a switch into a Product Management career but don’t know where to start from: this article is for you. Product Management roles started rising less than 20 years ago, it’s normal not having a clear understanding of what the role encompasses. I found this great definition from Atlassian that provides a fundamental understanding of the role. A product manager is the person who identifies the customer need and the larger business objectives that a product or feature will fulfill, articulates what success looks like for a product, and rallies a team to turn that vision into a reality. ( ) Sherif Mansour on Atlassian In this article, I provide simple steps and tools to step up in this thrilling career path. What you will find in this article: A complete understanding of what a PM is and is not The four types of Product Managers: Consumer, Internal, B2B, Technical The four qualities of a great PM: Communication, Technical knowledge, Problem-solving mindset and leadership Image source: paperblog Defining the role of a Product Manager (PM) 1) Product Manager roles are trending As this great from Productboard states, article “We are in the Golden Age of Product Management”. What was not a long time ago an obscure job title that none would know how to define is now one of the hottest roles in the Market. Product Management roles in the US have grown an astounding 32% from August 2017 to June 2019, which is five times more than the growth for overall jobs. ( from Neal Iyer from Product Management Insider). With some specific roles like Growth Product Manager showing a 425% increase in average monthly interest over the last 5 years in Google Trends ( ). Study ProductLed Image source: Neal Iyer on Medium Product Management roles in the US have grown 32%, which is five times more than the growth for overall jobs. Before getting tips on how to transition to a Product Management position, let’s define its role. 2) What a PM is not: 2.1) A PM is not a Manager. Let’s start with the least obvious, as the job title doesn’t say, a PM is not a manager (in the traditional term). His primary responsibility is not caring for a team and their growth but Product. As scrum.org put it, the Product Owner role is to maximize the Product Value. If you work as a PM, you quickly realize that if, of course, it implies having a motivated, high-performing team, so it’s in your best interest to having everyone feeling fulfilled about their job and passionate about their mission. 2.2) A PM is not an engineer. This misconception comes from the fact that most recruiters and most job posts require you to have a computer science degree. It’s particularly true for a specified type of PM that I outline later. Having a strong technical background is an excellent advantage for your job, helping you understand more how the Product is going to be implemented, and the limitation you could face. However, the PM is not an engineer as he decides on the What, not on the How. The PM determines what Product do we want to implement to maximize value for our users. The development team decides how to implement the Product and with which technology. The PM is not an engineer as he decides on the What, not on the How 2.3) A PM is not a designer. A good PM needs a good understanding of customer needs. Having a substantial User Experience (UX) background helps you tailor a product closer to the requirements of your users. However, as a PM, you don’t need to be proficient in Design and a pro of Adobe and Sketch. Depending on the organization and your role, you can delegate those responsibilities to UI and UX designers. Image source: Altexsoft : Digression — The difference between PM in FGMC vs Software company Haircolor — L’Oréal Paris) L’Oréal Paris In this section, I want to clarify the confusion with the traditional PM role of FGMCs which belong within the Marketing Department. You might have one friend working at L’Oréal as a PM of a Shampoo line who has no idea about Agile, UX, or the concept of maximizing value for the user. However, they can tell you all about the shampoo market, their last marketing campaign, and their retail strategy. This PM usually evolves as Product Line manager ( and then finally promoted to a Brand manager ( ). It’s a career path within Marketing, and although the PM shares the same job title, the role differs a lot from the Agile Product Manager, most frequently from Software companies, that we are focusing on in this article. 3) What is a PM: The PM is the person responsible for the success of the Product. Note that the Product can be an app, a line of products, a subsection of a product (a section of a website). What are some Product Manager Responsibilities: User Research Roadmapping Data analysis Project Management Strategy Stakeholder Management Cross-functional management Image source: Productcoalition 4) The four types of Product Managers There are four types of Product Managers roles. They differ according to the type of organization and Product. 4.1) Consumer PM He builds products for the general consumer. One example could be TickTock. His primary responsibilities are: Ideation Testing Data Analysis User research to prioritize work for new features 4.2) Internal PM: He builds products for users within their own company. For example, software used by Customer Service. His primary responsibilities are: Project Management Execution Not much design as it’s internal : 4.3) Business to Business (B2B) PM He builds products for use at another company. For example, a PM at HubSpot, an inbound marketing automation tool. The principal characteristics of B2B products are: Low volume, high cost, sold by sales teams There is a smaller number of end-users (compare to consumer products), although, for some successful companies like SalesForce, this number is still huge Execution focus The main challenge is that the product end-user (e.g., Sales) is different from the purchase decision-maker (e.g., Finance) : 4.4) Technical PM He focuses on technical product manager. For example, Google Product manager working in the search algorithm. Specificities of a Technical PM are: Can be a subtype of any of the other types Solving complex technology challenges I include an infographic from Mckinsey that offers a different categorization: Image Source: 5 Academyxi The four qualities of a great Product Manager We previously saw that there are different types of Product Manager, and each type requires different skills and backgrounds. However, some broads qualities considerably help to switch to a Product Manager position. : You can communicate via email, phone, or in-person meetings. As a PM, you have to talk with internal and external stakeholders frequently to gather feedback and express your vision. And remember the best communicators are the best listeners, or as the saying goes, “Seek First to Understand, Then to Be Understood. Communication : Having an engineering degree is not mandatory, but the more technical knowledge, the easier. As a PM, you need to understand the technology and to be able to talk to it with your team. Technical knowledge : A PM is a problem solver. He loves having complex challenges that require thinking outside the box. Being flexible and create is not a set of skills but a mindset. Problem-oriented : A PM is not a manager, but he is a leader. And a good leader makes sure to listen to the team and address concerns. He is open to criticism, always looking to improve his role and the Product. He practices transparency within the team and for the entire organization. He is accountable for the Product. He makes sure to give credit to others to create a motivated team. Leadership Image source: kent-teach Congratulations if you made it this far! I hope this article helped you to switch to a Product Manager position. If you are interested to learn more make sure to read Part 2 and Part 3 of the From Zero to Product Manager series. Please let me know in the comment section if there additional tips that you would find useful. And don’t forget to check my other posts about Product Management and Agile Methodologies.

What Product Managers do and How to become one [Part 1 — From Zero to Product Manager]

Want to know what it takes to make it in UX design? We recently asked four awesome women in UX about their own experiences in carving out a career in this exciting field. They offered a refreshingly honest and insightful look into the world of UX, with each taking a unique journey to get to where they are now. UX design, or user experience design, is all about taking a customer on a journey through a given medium — be it audio-visual, an interaction with an AI, or navigating a web page. It’s about invoking feelings from a user, and creating targeted stories for brands. It’s about unleashing your creativity to realise tangible, measurable results. What led Sharni Allen, Luisa Lombardo, Bettina Marson and Hannah Heffernan into a career in UX? What were their personal hurdles along the way? And what tips do they have for girls ( ) looking at a future in UX design? and guys Let’s find out what leading women in UX Design have to say. Sharni Allen — UX Designer & Service Designer, Freelance Sharni Allen is a former Academy Xi instructor who currently freelances as a leading UX and service designer. She describes herself as a ‘UX generalist’ who is comfortable facilitating client workshops as well as creating user journeys and wireframes. Sharni has worked as a UX lead at agencies such as whiteGREY and Webling Interactive . What’s your background, and how did you get into UX design? I had worked my way up through the digital production ranks for over 10 years, culminating in an executive producer role where I was overseeing large scale digital transformation projects. About six years ago I started to see a shift in the marketplace — businesses were demanding more expertise from agencies in user-centred design. They not only wanted to see the research behind our solutions, they wanted to be involved in the process. Coming from a traditional waterfall background, the opportunity to work in a more lean and agile way with multidisciplinary teams who were producing results backed by actual research and testing was too good to pass up. I took the skills I had already refined as a producer, such as personas, empathy maps and wire-framing, and put myself on the market as a UX architect. The rest is history. What did you study to get into UX? I did a back in the days when we were still burning CDs and designing websites in 256-colour palettes. The foundations of user-centred design have not changed; the were the industry leaders back then, and are still at the top of the reading list of every UX practitioner around the world. Bachelor of Multimedia Nielsen Norman Group My years in digital production gave me a lot of experience in researching and developing the artefacts required for problem definition and solutions. The other soft skills I needed to succeed came naturally; confidence in public speaking, backing up my decisions when challenged, collaboration and empathy. What are the biggest hurdles you’ve found in adapting to the world of UX design? Getting client buy-in on the process and committing the required amount of time to work collaboratively. Enthusiasm is always high at the start of the project discovery. But continuing through the research, empathy and ideation phases of a project can require an incredible commitment of time from client stakeholders, who may become more hands off and ‘trust us’ — we’re the experts, aren’t we? Decisions can’t be made in silos. There needs to be a consensus on decisions made continuously through the project. Ensuring your decision-makers (product owners, SMEs, etc.) are taken on the journey with you is critical to project success. What do you wish you knew when you started? What would you do differently if you had your time again? I’d buy more Bitcoin! But seriously, taking a step back from the digital tools available and putting things up on a wall is the best way to start every single day. Having a project Kanban (Agile tool) and a daily stand up with the team can raise problems early and often. It fosters team spirit and and an understanding of the project as a whole, rather than the small piece each team member may be working on. Any tips for people interested in pursuing a career in UX design? The hard skills can be taught, the soft skills must be learned. The UX business is not for the shy or timid. It is all about the user — getting out there and talking to the audience, empathising with their needs and wants and then presenting back your findings is a big part of the job. You may be questioned by those who demand answers, challenged by those who don’t believe your findings, and proved completely wrong about your assumptions. UX is a learning exercise in itself, and without those lessons we can’t learn from the mistakes of the past and deliver useful products and services. Luisa Lombardo — Product Designer, hipages Luisa Lombardo is currently engaged as a product designer at hipages , a revolutionary Australian home improvement marketplace. Luisa made her start as a graphic designer in Sydney, before enjoying adventure and career-growth in the UK, and finally returning home as a senior. A switch to UX design came next, as it offered a fresh professional challenge while reinvigorating her passion for design. What’s your background, and how did you get into UX design? My background was in graphic design, where I focused on branding, advertising and print. Although I loved this space, I felt I needed a new challenge in my career and saw an opportunity to improve my skill set and move into UX design. What did you study to get into UX? Most of my UX learning has been on the job with hands-on experience. I have been very fortunate to work with some great UX designers who have been great mentors for me. I also completed a digital design program at Tractor Design School (now sadly closed) to help me make the transition. What are the biggest hurdles you’ve found in adapting to the world of UX design? Having the confidence to get into UX sooner! It’s always daunting making that first big leap in changing careers and starting from the beginning. What do you wish you knew when you started? What would you do differently if you had your time again? I wish I knew that my skills as a graphic designer would give me a great foundation in the UX world. It has made the change easier than I thought. Any tips for people interested in pursuing a career in UX design? Never stop learning. Read UX articles from sites like and . I like to start my day by reading the latest blog post and getting inspired by the work that others are doing. Also get out there and meet people in the UX community. Having contacts in the industry is a great way to build your career. Medium Muzli Bettina Marson — UX Specialist, Telstra Bettina Marson is a multidisciplinary creative. An award-winning artist and writer, she has more recently put her talents to work in the field of UX. Bettina is currently a UX & UI designer in the digital products & services department at Telstra , working in customer success management in the rapid delivery automation team. What’s your background, and how did you get into UX design? I’ve always been attracted to creative pursuits and technology. Initially I studied a , which led to a career working as a concept artist and environment designer. During that time I cultivated an interest in working closer with technologies to improve the lives of other people — both creating engaging experiences and solving real world problems. Bachelor of Game Design This led me to enroll in my master’s degree to fine-tune my focus and pursue user experience and interaction design as a career. What did you study to get into UX? I studied a . During my studies I created interactive prototypes and user experiences for use in the health, education, communication and lifestyle industries, which I found to be incredibly rewarding. Master of Interaction Design at Monash University What are the biggest hurdles you’ve found in adapting to the world of UX design? With the rate that technology is being developed and adopted, one of the greatest challenges I’ve found is that you need to not only be knowledgeable in UX practices, but also in the latest technologies and their limitations. What do you wish you knew when you started? What would you do differently if you had your time again? When you think of user experience design, you immediately think of interfaces and excellent customer service. User experience design is more diverse than that; we live in a world with many different technologies that allow for many different kinds user experiences, such as virtual reality and voice-based interaction as seen in products like Amazon’s Alexa and Google Home. User experience is not purely visual, and we must remember that people interact with technology in different ways. It’s important to not only keep up to date with the rapidly changing world of technology, but to ensure that the experiences you are creating are inclusive, intuitive and a delight to use. Any tips for people interested in pursuing a career in UX design? If you’re interested in this rewarding and challenging field, I definitely recommend attending a workshop or meet-up to find out first-hand what it’s like to work as a UX specialist. There are also many fantastic online resources to upskill in UX design if you’re looking to bring user-centric practices into your existing work. Hannah Heffernan — Product Designer, Canva Hannah Heffernan loves to create, and has managed to make a successful career out of this passion. During a stint in Berlin she also obtained a taste for startup life. And where better to be a startup loving creative than Australia’s billion dollar startup baby, Canva . What’s your background, and how did you get into UX design? I started out in advertising agencies, working first as a illustrator and designer, then as an art director. I came to UX while living in Berlin five years ago. My limited German didn’t get me far in traditional agencies, but the tech and startup scene was really strong. That’s where the demand was, plus tech tends to look further for talent so naturally the teams are very international and English becomes the default for everyone. My first role was as a real generalist in a small start-up. This is where I got the bug for designing for products. What did you study to get into UX? Like many of us, I didn’t. I studied and majored in illustration. My illustration background has definitely helped me design products. communication design What are the biggest hurdles you’ve found in adapting to the world of UX design? I made the shift to UX with a new job in a new company after maternity leave. That was a hurdle in itself! I found the biggest challenge was shifting from designing intuitively to having to inform my decisions with data. Design has always been emotive for me, and I guess my decision making is pretty gut-feel as well. Having the discipline to make more objective decisions and validating my hunches has taken practice. It is still something I have to practice! What do you wish you knew when you started? What would you do differently if you had your time again? I wish I knew that I already had all the skills, I just needed to apply them differently! Art direction and designing for advertising is about distilling an idea down to its simplest form, empathising with consumers and getting to the heart of what they’re trying to achieve in their lives and how your product can help them. To me, this is UX. Just the output is different. As women, we’re often not very good at selling how awesome we are! If I could go back to before those first terrifying UX role interviews, I’d practice being a better salesperson! It felt like this new scary industry that I knew nothing about, but I actually had 10 years of really relevant experience. My thinking was that I had to find a company willing to give me a break, but they actually would have been lucky to have me! Any tips for people interested in pursuing a career in UX design? If you already work in a creative field, or have any problem-solving element to what you do now, you’re probably already doing some UX! Find opportunities to apply tried and true UX methodologies and processes to projects you’re working on now. Then, when you make the leap to a UX role, you’ll already have the tools to get going. Is it time you made your start in the world of user experience design, just like these trailblazing women in UX? Check out The Martec’s other UX articles or for more info. The Future of a Career in UX The Pros and Cons of Bootcamps for UX Designers

UX Design Insights from 4 Women in Tech

. There’s an argument to say that collecting experiences is the meaning of life; that seeing new sights and feeling new feelings allows an individual to understand the breadth of the human experience, and to thus live life to its fullest. Human experiences It’s from this lofty (and perhaps ) perch that we’re going to survey the world of UX design, because it is the role of the UX designer to craft the experiences of a digital product’s users. They control the usability, accessibility and enjoyment of the UX (user experience), and have the final say on whether it is to be one that is pleasant, fulfilling, jaw-dropping, or indeed . slightly conceited terrible Sculpting a digital world is an exciting prospect, and has seen the role of UX designer becoming ever more desirable in recent years. And while we at have previously spoken about and , we are yet to guide UX designers through that middle period of the role — the crucial developmental stage that sees many a fresh-faced UXer either sink or swim. The Martec how to kick off a career in UX how to navigate the very top rungs of the profession In search of answers we spoke to three UX design — , a UX Specialist at , a Product Designer at , and , Lead UX Design Instructor at — to hear how they managed to navigate these often choppy waters. dab hands Bettina Marson Telstra Hannah Heffernan Canva Robert Williams Academy Xi What do the pros have to say about forging a path in UX design? Let’s find out. Understanding whether a UX design career is right for you Many a budding UX designer has been surprised at what the role entails once the choice was made to pursue the career. Understanding whether you’re a good fit for the role is the first step on the road to UX design success. “I’ve always been attracted to creative pursuits and technology,” says Marson. Her entry into the UX space wasn’t particularly conventional, but it was one that seems to have set her up for success. “Initially I studied a Bachelor of Game Design, which led to a career working as a concept artist and environment designer. During that time I cultivated an interest in working closer with technologies to improve the lives of other people — both creating engaging experiences and solving real world problems. This led me to enrol in my master’s degree to fine-tune my focus and pursue UX and interaction design as a career.” While Marson evolved a passion for all things UX, for others this enthusiasm is inherent. So, are you a natural-born UX designer? One way to find out is to look at the skills that separate UXers from other professionals, and see if your strengths align. The key skills that make a UX designer exceptional So what do our experts see as the talents shared by the top UX designers? As a UX professional at design software provider Canva, Heffernan feels that being able convey your ideas through sketching is a skill that separates the best UX designers from the rest. “I may be biased as someone with a background in art and illustration, but being able to visually share an idea in seconds with a scribble has helped me immensely. With just pen and paper I can jam with my product manager or an engineer and figure out an idea together. It’s so much more collaborative than making wireframes or mock-ups on my computer.” “Communication,” Heffernan adds when pressed for more skills. “I’ve worked with some otherwise brilliant UX researchers and designers who struggled to get key people on board because they didn’t know how to [communicate]. You can’t operate as this elite theoretical specialist and expect others to agree when you present your recommendations. Get people invested in UX early, and make it accessible, practical and inclusive for people who may be new to it.” Williams agrees. “In UX design, soft skills have a huge role. The ability to communicate well — to facilitate, to organise and to collaborate — are hallmarks of an exceptional UX designer.” He also suggests that an exceptional UX designer needs to be good at identifying what creates a better world for the customer, and must understand that it takes courage to create change. Rather than focusing on learnt skills, Marson instead identifies the key traits of an A-grade UX designer. She sees adaptability and drive as the two most important qualities that the best professionals in the field share, as it’s . “With the rate [at which] technology is being developed and adopted, one of the greatest challenges I’ve found is that you need to not only be knowledgeable in UX practices, but also in the latest technologies and their limitations.” a profession that is constantly evolving Which seamlessly brings us to the next chapter of our pro’s guide to UX design… Approaches to upskilling, and resources that help “There are so many schools of thought and methodologies [in UX design] to pull from,” notes Heffernan. In such a diverse field it’s impossible to stay abreast of everything. She recommends that you instead update your skillset in response to project needs. With that in mind, Heffernan advises the following upskilling approaches and resources for UX designers who are looking to be the best that they can be: Become a part of product communities with . These events really helped when I was new to Sydney’s tech space, and often cost no more than a glass of wine! meetup.com I read years ago, but it really stuck with me. It was my first introduction into design thinking, before it was a term that I’d come across. Glimmer by Warren Berger by Don Norman is a really interesting study of usability, also a great read if you’re just a nerd for beautiful design. The Design of Everyday Things by Robert Hoekman Jr. was a book that (Canva founder) Mel recommended when I first started. It taught me some useful things about how Canva thinks about product design. Designing the Obvious by Steve Krug is a favourite. I always revisit it for practical tips — monthly at least! Krug’s website also has some really great downloadable resources. Don’t Make Me Think! Advanced Common Sense In the same vein, Williams warns against the urge to be a jack of all trades, and recommends that you instead become a master of one (or perhaps a handful). “Know what you do well. We have so many opportunities to flex our talent in different directions, so it’s worth recognising your specialty. Are you a brilliant researcher? Take your insights into forming a series of objectives within a prototype! Are you fantastic at designing and creating user journeys? Head out with the research team sometime to hear the customer voice directly!” For those into literature, Williams echoes Heffernan’s sentiments on Steve Krug, and also lists the following books as required reading for UX designers: by Eric Reiss Usable Usability by Jesmond J. Allen and James J. Chudley Smashing UX Design by Andy Polaine Service Design: From Insight to Implementation by Jeff Gothelf Lean UX Williams also advises that budding UXers check out the online resources provided by the organisation that he calls ‘the founders of UX’ . — Nielsen Norman Group Lessons from those who have done it all before What else should UX designers know about their profession? Having spent years in the field, our experts have seen it all before. So what learnings and advice would they pass on to those who are trying to make it in the world of UX? Marson goes first. “When you think of user experience design, you immediately think of interfaces and customer service. But UX design is more diverse than that. We live in a world with many different technologies that allow for many different kinds user experiences, such as virtual reality and voice-based interaction as seen in products like Amazon’s Alexa and Google Home. User experience is not purely visual, and we must remember that people interact with technology in different ways. It’s important to not only keep up to date with the rapidly changing world of technology, but ensure that the experiences you are creating are inclusive, intuitive and a delight to use.” What has Heffernan learned about UX design over the course of her career? “That it means something different in every company! Some businesses require a more research- and strategy-based specialist. Other roles I’ve had or interviewed for might talk UX, but then you figure out it’s mostly a UI design position.” In short, she says, you need to be prepared for anything — “it just covers such a broad set of skills.” “In my experience, I’ve seen that the majority of our work is intangible, and often invisible to the end user,” notes Williams. “The user may only see a view of the outcomes in a single feature, button or tool we provide to them.” So as a UX designer, he says it’s not ‘what’ is created, but ‘how’ — “ we work within a project group, we collaborate with others and communicate our findings and deliverables” — that is most important. how how A role on the rise The demand for UX designers isn’t going to decrease any time soon. With so many human experiences now turning digital, organisations are always going to need specialists to help bring their UX dreams to reality. Those charged with crafting user experiences need to give themselves the best user experience too, only in a professional sense. Gain guidance from more experienced contemporaries. Learn from their mistakes. Identify your strengths and play to them. Motivate yourself to upskill. Do all this and you’ll be well on your way to becoming one of the most exceptional UX designers in your field. Are you looking to take the next step in your UX design or tech career? Great news — the biggest names in Australian and international tech are looking for talent right now! Check out our for the latest open positions. job boards

The Pro’s Guide to UX Design

The rapid growth of AI is transforming many areas of life, including education, with ChatGPT at the forefront of this change. Recently, OpenAI, the company behind ChatGPT, introduced ChatGPT Edu—a tailored version of its flagship AI tool built for universities to “responsibly” deploy AI to students, faculty, researchers, and campus operations. Riding on the success of its ChatGPT Edu in Universities like the University of Oxford, Wharton School of the University of Pennsylvania(opens in a new window), University of Texas at Austin, Arizona State University(opens in a new window), and Columbia University in the City of New York, ChatGPT edu comes with a lot of promise— one that can help universities integrate AI into every aspect concerning students and faculties. But as always, with every ease comes hardship, and ChatGPT Edu too can be a double-edged sword. But we’ll discuss the downsides a little later. Let’s lead with how ChatGPT Edu can be constructive for higher educational institutions, their students, and faculties. Beginning with the fact that ChatGPT leverages AI, it shouldn’t be a debate that its Edu version can provide faculty members and students with a satisfying level of personalization. By tailoring content and study material as per every student’s need, it can be the silver bullet that universities desperately need. With faculties and mentors grappling with the grueling task of managing the rising number of students in a class, personalization through AI can bring tremendous results for them. The fact that it can identify knowledge gaps and adapt to different learning styles symbolizes hope for the future of academia. AI has always attracted keen minds and ChatGPT Edu, given its inherent character to integrate AI with interactive activities, can make learning more engaging and interactive. Students who are disengaged from routine lectures and other pedagogical processes may find AI’s use in education to be an attractive prospect. This can help keep the students more engaged– a challenge that is commonly observed among students. They are highly disoriented, given the fact that a vast majority of higher educational institutions are still sticking to rote learning practices. Enter ChatGPT Edu, by engaging students in simulated discussions, role-playing scenarios, and critical thinking skills, it can help them stay focused and better oriented, and help suitably streamline their academic priorities. For students with learning disabilities, ChatGPT Edu can be the holy grail. It can offer personalized feedback in various formats, including audio or visual aids, catering to different learning preferences. ChatGPT Edu can also help universities stay clear of the language barriers that often arise when students and faculties of one country inhabit the other. Offering real-time language translation and support can allow students to participate more actively in classroom discussions and access course materials in their preferred language. What’s beyond this is it can also assist in personalized pronunciation practice and real-time feedback, fostering greater confidence and fluency for non-native speakers. The faculty members, moreover, can use the translation feature to ensure all students understand the course material the way they want them to. They can translate lectures, presentations, and handouts, allowing students to follow along and engage more effectively by harnessing the tool properly. We’ll have to see how ChatGPT’s Edu version performs once it is widely circulated in the public domain. For students from diverse backgrounds who face difficulty understanding concepts and things, it can be a big breather as it can translate complex concepts and methods into simpler language. Being able to offer text-to-speech conversion, and the capability to generate visual aids and summaries will surely be a cherry on the cake for both students and faculties alike. Lastly, what seems to be interesting from the faculty’s perspective is that it can offer continued assistance to faculty research. It can certainly help analyze vast amounts of data, generate research hypotheses, and streamline literature reviews, freeing up valuable time for more focused research endeavors. AI in education isn’t a capricious idea anymore. The downsides One of the biggest concerns arising out of the use of AI in educational institutions is there is a chance of students getting too reliant on it for everything. This can have serious repercussions. They can lose touch with realistic teaching pedagogy and become more AI-dependent thereby stepping into that alarming “virtual” zone of learning, where anything without the aid of AI is less understood or best, misunderstood. An equally concerning aspect is the fact it can severely hinder critical thinking and problem-solving skills for a lot of the students. Secondly, it’s believed AI is only as good as the data it's trained on. If that data is biased or fed with wrong inputs, ChatGPT Edu could perpetuate stereotypes or deliver inaccurate information—something which can lead the learner astray, thereby inhibiting the whole learning experience and marring the learning curve. Lastly, overuse of this technology could blur the lines between original and duplicate content (read plagiarism). Rightly so! Plagiarism has been a serious concern for academic institutions, as cases of AI-blended learning have risen over the years. And these concerns of AI and plagiarism are legitly myriad. We’ll have to see how this tailor-made ChatGPT tool can minimize incidents of students and learners plagiarizing content in universities. Summing Up Education offers incredible potential for the application of AI technologies. With the introduction of ChatGPT Edu and its subsequent integration into educational institutions across all research and learning touchpoints, it seems the educational landscape may be in for some unanticipated and enormous change. Nevertheless, learning the tricks of AI-based learning practices and dupery that could be possibly built around it by mala fide learners, can certainly be a solution and shouldn’t be taken for granted. While on one hand, adopting regular audits and data curation practices can help minimize bias and misinformation, on the other, universities can design courses that blend ChatGPT Edu for personalized learning while prioritizing interactive activities and discussions that are likely to produce the maximum output while remaining efficient. With the integration of this innovative tool, learning practices bear a likelihood of showing improved outcomes but it also brings with itself a multitude of vulnerabilities and risks. It just remains to be seen if post its integration with universities on a massive scale, whether it will endanger institutions and recede learning to a dangerous low or will it push towards an academic environment that appears technologically and educationally utopian. The rapid growth of AI is transforming many areas of life, including education, with ChatGPT at the forefront of this change. Recently, OpenAI, the company behind ChatGPT, introduced ChatGPT Edu—a tailored version of its flagship AI tool built for universities to “responsibly” deploy AI to students, faculty, researchers, and campus operations. Riding on the success of its ChatGPT Edu in Universities like the University of Oxford, Wharton School of the University of Pennsylvania(opens in a new window), University of Texas at Austin, Arizona State University(opens in a new window), and Columbia University in the City of New York, ChatGPT edu comes with a lot of promise— one that can help universities integrate AI into every aspect concerning students and faculties. But as always, with every ease comes hardship, and ChatGPT Edu too can be a double-edged sword. But we’ll discuss the downsides a little later. the success of its ChatGPT Edu in Universities Let’s lead with how ChatGPT Edu can be constructive for higher educational institutions, their students, and faculties. Beginning with the fact that ChatGPT leverages AI , it shouldn’t be a debate that its Edu version can provide faculty members and students with a satisfying level of personalization. By tailoring content and study material as per every student’s need, it can be the silver bullet that universities desperately need. With faculties and mentors grappling with the grueling task of managing the rising number of students in a class, personalization through AI can bring tremendous results for them. The fact that it can identify knowledge gaps and adapt to different learning styles symbolizes hope for the future of academia. ChatGPT leverages AI ChatGPT leverages AI personalization through AI personalization through AI AI has always attracted keen minds and ChatGPT Edu, given its inherent character to integrate AI with interactive activities, can make learning more engaging and interactive. Students who are disengaged from routine lectures and other pedagogical processes may find AI’s use in education to be an attractive prospect. This can help keep the students more engaged– a challenge that is commonly observed among students. They are highly disoriented, given the fact that a vast majority of higher educational institutions are still sticking to rote learning practices. Enter ChatGPT Edu, by engaging students in simulated discussions, role-playing scenarios, and critical thinking skills, it can help them stay focused and better oriented, and help suitably streamline their academic priorities. For students with learning disabilities, ChatGPT Edu can be the holy grail. It can offer personalized feedback in various formats, including audio or visual aids, catering to different learning preferences. ChatGPT Edu can also help universities stay clear of the language barriers that often arise when students and faculties of one country inhabit the other. Offering real-time language translation and support can allow students to participate more actively in classroom discussions and access course materials in their preferred language. What’s beyond this is it can also assist in personalized pronunciation practice and real-time feedback, fostering greater confidence and fluency for non-native speakers. The faculty members, moreover, can use the translation feature to ensure all students understand the course material the way they want them to. They can translate lectures, presentations, and handouts, allowing students to follow along and engage more effectively by harnessing the tool properly. We’ll have to see how ChatGPT’s Edu version performs once it is widely circulated in the public domain. translation feature translation feature For students from diverse backgrounds who face difficulty understanding concepts and things, it can be a big breather as it can translate complex concepts and methods into simpler language. Being able to offer text-to-speech conversion, and the capability to generate visual aids and summaries will surely be a cherry on the cake for both students and faculties alike. Lastly, what seems to be interesting from the faculty’s perspective is that it can offer continued assistance to faculty research. It can certainly help analyze vast amounts of data, generate research hypotheses, and streamline literature reviews, freeing up valuable time for more focused research endeavors. AI in education isn’t a capricious idea anymore. The downsides One of the biggest concerns arising out of the use of AI in educational institutions is there is a chance of students getting too reliant on it for everything. This can have serious repercussions. They can lose touch with realistic teaching pedagogy and become more AI-dependent thereby stepping into that alarming “virtual” zone of learning, where anything without the aid of AI is less understood or best, misunderstood. An equally concerning aspect is the fact it can severely hinder critical thinking and problem-solving skills for a lot of the students. Secondly, it’s believed AI is only as good as the data it's trained on. If that data is biased or fed with wrong inputs, ChatGPT Edu could perpetuate stereotypes or deliver inaccurate information—something which can lead the learner astray, thereby inhibiting the whole learning experience and marring the learning curve. Lastly, overuse of this technology could blur the lines between original and duplicate content (read plagiarism). Rightly so! Plagiarism has been a serious concern for academic institutions, as cases of AI-blended learning have risen over the years. And these concerns of AI and plagiarism are legitly myriad. We’ll have to see how this tailor-made ChatGPT tool can minimize incidents of students and learners plagiarizing content in universities. AI and plagiarism AI and plagiarism Summing Up Education offers incredible potential for the application of AI technologies. With the introduction of ChatGPT Edu and its subsequent integration into educational institutions across all research and learning touchpoints, it seems the educational landscape may be in for some unanticipated and enormous change. incredible potential for the application incredible potential for the application Nevertheless, learning the tricks of AI-based learning practices and dupery that could be possibly built around it by mala fide learners, can certainly be a solution and shouldn’t be taken for granted. While on one hand, adopting regular audits and data curation practices can help minimize bias and misinformation, on the other, universities can design courses that blend ChatGPT Edu for personalized learning while prioritizing interactive activities and discussions that are likely to produce the maximum output while remaining efficient. With the integration of this innovative tool, learning practices bear a likelihood of showing improved outcomes but it also brings with itself a multitude of vulnerabilities and risks. It just remains to be seen if post its integration with universities on a massive scale, whether it will endanger institutions and recede learning to a dangerous low or will it push towards an academic environment that appears technologically and educationally utopian.

ChatGPT Edu: Dawn of a Brighter Academic Future, or a Slippery Slope Towards Diminished Learning?

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract The emergent abilities of Large Language Models (LLMs), which power tools like ChatGPT and Bard, have produced both excitement and worry about how AI will impact academic writing. In response to rising concerns about AI use, authors of academic publications may decide to voluntarily disclose any AI tools they use to revise their manuscripts, and journals and conferences could begin mandating disclosure and/or turn to using detection services, as many teachers have done with student writing in class settings. Given these looming possibilities, we investigate whether academics view it as necessary to report AI use in manuscript preparation and how detectors react to the use of AI in academic writing. 1 Introduction There is both excitement and concern about the impact that artificial intelligence (AI) could have on our world. In academia, this has inspired an explosion of research around AI. Some researchers are leveraging AI to help in answering research questions; for example, as a tool for performing statistical or textual analyses (e.g., Mullainathan and Spiess 2017, Athey and Imbens 2019, Fudenberg and A. Liang 2019, Farrell, T. Liang, and Misra 2020, Rambachan et al. 2021, Björkegren, Blumenstock, and Knight 2022, Capra, Gomies, and Zhang 2023, Franchi et al. 2023, Salah, Al Halbusi, and Abdelfattah 2023), or for the design or implementation of experiments (e.g., Beck et al. 2020, Charness, Jabarian, and List 2023, Horton 2023). Others have focused more on the interaction of humans with AI systems (e.g., Dietvorst, Simmons, and Massey 2015, Deza, Surana, and Eckstein 2019, Gajos and Mamykina 2022, Steyvers et al. 2022, Sundar and Lee 2022, Tejeda et al. 2022, Wang, C. Liang, and Yin 2023, Yang, M. Palma, and A. Drichoutis 2023) and the larger societal ramifications of AI (e.g., Agrawal, Gans, and Goldfarb 2019, Lambrecht and Tucker 2019, Obermeyer et al. 2019, Yang, M. A. Palma, and A. C. Drichoutis 2023, Chien et al. 2020, Rolf et al. 2020, Zuiderwijk, Chen, and Salem 2021, Pallathadka et al. 2023, Ray 2023, H. Singh and A. Singh 2023). In terms of societal ramifications, one topic of interest is how AI will impact academia itself, especially given the emergent abilities of Large Language Models (LLM), which power tools like ChatGPT and Bard. Most of this research has centered on student use of AI to complete course or degree requirements (e.g., Cowen and Tabarrok 2023, Daun and Brings 2023, Fyfe 2023, Ibrahim et al. 2023, Jungherr 2023, Malik et al. 2023, Schmohl et al. 2020, Shahriar and Hayawi 2023). However, there has been less focus on the impact that AI is having and might have on manuscript preparation for academic journals, even though the impact this could have on science, and the propagation of scientific results, is potentially large. An exception is Korinek (2023), who documents several use cases for LLMs for researchers, including academic writing. He notes, “LLMs can edit text for grammatical or spelling mistakes, style, clarity, or simplicity.” As a result, this class of tools “allows researchers to concentrate their energy on the ideas in their text as opposed to the writing process itself.” In particular, “this set of capabilities are perhaps most useful for non-native speakers who want to improve their writing.” The potential impact of these tools is validated by the emergence of companies that help academics leverage LLM in their writing. Some of those initiatives are even run by professors from academia, such as online workshops that teach researchers how to use ChatGPT for academic publishing. However, an emerging set of papers (e.g., Altmäe, Sola-Leyva, and Salumets 2023, Thorp 2023, Shahriar and Hayawi 2023 and Hill-Yardin et al. 2023) detail some practical and ethical concerns with the use of these tools in preparing manuscripts for academic journals. For example, AI can generate a text with mistakes, including incorrect math, reasoning, logic, factual information, and citations (even producing references to scientific papers that do not exist). There are many welldocumented examples where LLM’s “hallucinate” and provide completely fictitious information. On top of this, these tools may also produce text that is biased against particular groups. These issues are exacerbated by the “black box” nature of LLM suggestions, meaning that we lack an understanding of how they work (e.g., Bommasani et al. 2021). Given these issues, liabilities arise when authors submit papers without fully vetting the text generated by LLMs. In addition, given that they are trained on a corpus of other writing and not the author’s own writing, using the output of tools like ChatGPT and Bard without proper attribution could be considered plagiarism. Because of these concerns, authors may voluntarily choose to disclose whether they have used LLMs in preparing their manuscripts, as suggested by Bom (2023). However, given well-documented failures of disclosure in the field and lab (e.g., Dranove and Jin 2010, Jin, Luca, and Martin 2015), journals, conferences, and associations may respond by mandating disclosure through reporting requirements, as is already done for conflict-of-interest issues. For example, Elsevier limits the use of AI by authors “only to improve the language and readability of their paper” and requires “the appropriate disclosure... at the bottom of the paper in a separate section before the list of references.” See Table 6 in Appendix B for a list of major publishers who have voluntary and mandatory disclosure policies for AI use. To enforce these reporting requirements, journals, conferences, and associations might turn to using detection services, as many teachers have done with student writing in class settings. The use of LLMs in writing is potentially detectable because as Korinek (2023) notes, “Some observe that [LLM writing] is, naturally, a bit sterile and lacks the idiosyncrasies and elements of surprise that characterize human writing – a feature that detectors of LLM-written text zero in on.” Given these looming possibilities, we investigate whether academics view reporting AI use in manuscript preparation as necessary and how detectors react to the use of AI in manuscript preparation. We first conducted a survey of academics in which we elicited perceptions about reporting assistance in manuscript preparation. We then used GPT-3.5 to revise abstracts from the past 10 years of Management Science by asking the software program to either fix grammar or rewrite text. Finally, we ran the original abstracts and revised abstracts through a leading paid AI detection service. We have three main findings. First, as shown partially in Figure 1, the academics we surveyed were less likely to think that using AI to fix the grammar in manuscripts should be reported than using AI to rewrite manuscripts, but detection software did not always draw this distinction, as abstracts for which GPT-3.5 was used to fix grammar were often flagged as having a high chance of being written by AI. Second, we found little difference in preferences for reporting ChatGPT and research assistant (RA) help, but significant differences in reporting preferences between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Third, we found disagreements among the academics we surveyed on whether using ChatGPT to rewrite text needs should be reported, and differences were related to perceptions of ethics, academic role, and English language background. The rest of the paper is structured as follows. Section 2 introduces our survey design and our method for testing ChatGPT use with AI detection algorithms. Section 3 shows our main results. Finally, section 4 discusses some limitations of our work, open questions arising from our results, and suggestions for future research. Figure 1: Reporting views (a) vs. detection results (b). This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract The emergent abilities of Large Language Models (LLMs), which power tools like ChatGPT and Bard, have produced both excitement and worry about how AI will impact academic writing. In response to rising concerns about AI use, authors of academic publications may decide to voluntarily disclose any AI tools they use to revise their manuscripts, and journals and conferences could begin mandating disclosure and/or turn to using detection services, as many teachers have done with student writing in class settings. Given these looming possibilities, we investigate whether academics view it as necessary to report AI use in manuscript preparation and how detectors react to the use of AI in academic writing. 1 Introduction There is both excitement and concern about the impact that artificial intelligence (AI) could have on our world. In academia, this has inspired an explosion of research around AI. Some researchers are leveraging AI to help in answering research questions; for example, as a tool for performing statistical or textual analyses (e.g., Mullainathan and Spiess 2017, Athey and Imbens 2019, Fudenberg and A. Liang 2019, Farrell, T. Liang, and Misra 2020, Rambachan et al. 2021, Björkegren, Blumenstock, and Knight 2022, Capra, Gomies, and Zhang 2023, Franchi et al. 2023, Salah, Al Halbusi, and Abdelfattah 2023), or for the design or implementation of experiments (e.g., Beck et al. 2020, Charness, Jabarian, and List 2023, Horton 2023). Others have focused more on the interaction of humans with AI systems (e.g., Dietvorst, Simmons, and Massey 2015, Deza, Surana, and Eckstein 2019, Gajos and Mamykina 2022, Steyvers et al. 2022, Sundar and Lee 2022, Tejeda et al. 2022, Wang, C. Liang, and Yin 2023, Yang, M. Palma, and A. Drichoutis 2023) and the larger societal ramifications of AI (e.g., Agrawal, Gans, and Goldfarb 2019, Lambrecht and Tucker 2019, Obermeyer et al. 2019, Yang, M. A. Palma, and A. C. Drichoutis 2023, Chien et al. 2020, Rolf et al. 2020, Zuiderwijk, Chen, and Salem 2021, Pallathadka et al. 2023, Ray 2023, H. Singh and A. Singh 2023). In terms of societal ramifications, one topic of interest is how AI will impact academia itself, especially given the emergent abilities of Large Language Models (LLM), which power tools like ChatGPT and Bard. Most of this research has centered on student use of AI to complete course or degree requirements (e.g., Cowen and Tabarrok 2023, Daun and Brings 2023, Fyfe 2023, Ibrahim et al. 2023, Jungherr 2023, Malik et al. 2023, Schmohl et al. 2020, Shahriar and Hayawi 2023). However, there has been less focus on the impact that AI is having and might have on manuscript preparation for academic journals, even though the impact this could have on science, and the propagation of scientific results, is potentially large. An exception is Korinek (2023), who documents several use cases for LLMs for researchers, including academic writing. He notes, “LLMs can edit text for grammatical or spelling mistakes, style, clarity, or simplicity.” As a result, this class of tools “allows researchers to concentrate their energy on the ideas in their text as opposed to the writing process itself.” In particular, “this set of capabilities are perhaps most useful for non-native speakers who want to improve their writing.” The potential impact of these tools is validated by the emergence of companies that help academics leverage LLM in their writing. Some of those initiatives are even run by professors from academia, such as online workshops that teach researchers how to use ChatGPT for academic publishing. However, an emerging set of papers (e.g., Altmäe, Sola-Leyva, and Salumets 2023, Thorp 2023, Shahriar and Hayawi 2023 and Hill-Yardin et al. 2023) detail some practical and ethical concerns with the use of these tools in preparing manuscripts for academic journals. For example, AI can generate a text with mistakes, including incorrect math, reasoning, logic, factual information, and citations (even producing references to scientific papers that do not exist). There are many welldocumented examples where LLM’s “hallucinate” and provide completely fictitious information. On top of this, these tools may also produce text that is biased against particular groups. These issues are exacerbated by the “black box” nature of LLM suggestions, meaning that we lack an understanding of how they work (e.g., Bommasani et al. 2021). Given these issues, liabilities arise when authors submit papers without fully vetting the text generated by LLMs. In addition, given that they are trained on a corpus of other writing and not the author’s own writing, using the output of tools like ChatGPT and Bard without proper attribution could be considered plagiarism. Because of these concerns, authors may voluntarily choose to disclose whether they have used LLMs in preparing their manuscripts, as suggested by Bom (2023). However, given well-documented failures of disclosure in the field and lab (e.g., Dranove and Jin 2010, Jin, Luca, and Martin 2015), journals, conferences, and associations may respond by mandating disclosure through reporting requirements, as is already done for conflict-of-interest issues. For example, Elsevier limits the use of AI by authors “only to improve the language and readability of their paper” and requires “the appropriate disclosure... at the bottom of the paper in a separate section before the list of references.” See Table 6 in Appendix B for a list of major publishers who have voluntary and mandatory disclosure policies for AI use. To enforce these reporting requirements, journals, conferences, and associations might turn to using detection services, as many teachers have done with student writing in class settings. The use of LLMs in writing is potentially detectable because as Korinek (2023) notes, “Some observe that [LLM writing] is, naturally, a bit sterile and lacks the idiosyncrasies and elements of surprise that characterize human writing – a feature that detectors of LLM-written text zero in on.” Given these looming possibilities, we investigate whether academics view reporting AI use in manuscript preparation as necessary and how detectors react to the use of AI in manuscript preparation. We first conducted a survey of academics in which we elicited perceptions about reporting assistance in manuscript preparation. We then used GPT-3.5 to revise abstracts from the past 10 years of Management Science by asking the software program to either fix grammar or rewrite text. Finally, we ran the original abstracts and revised abstracts through a leading paid AI detection service. We have three main findings. First, as shown partially in Figure 1, the academics we surveyed were less likely to think that using AI to fix the grammar in manuscripts should be reported than using AI to rewrite manuscripts, but detection software did not always draw this distinction, as abstracts for which GPT-3.5 was used to fix grammar were often flagged as having a high chance of being written by AI. Second, we found little difference in preferences for reporting ChatGPT and research assistant (RA) help, but significant differences in reporting preferences between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Third, we found disagreements among the academics we surveyed on whether using ChatGPT to rewrite text needs should be reported, and differences were related to perceptions of ethics, academic role, and English language background. The rest of the paper is structured as follows. Section 2 introduces our survey design and our method for testing ChatGPT use with AI detection algorithms. Section 3 shows our main results. Finally, section 4 discusses some limitations of our work, open questions arising from our results, and suggestions for future research. Figure 1: Reporting views (a) vs. detection results (b). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

AI Use in Manuscript Preparation for Academic Journals: Abstract and Introduction

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals A Survey Screenshots B Additional Tables Table 3 provides summary statistics for whether authors should acknowledge different forms of assistance. The top panel provides the average responses broken out by being a native English speaker or not, and the bottom panel by being a professor or not. Table 4 shows the results of asking if it is unethical to use ChatGPT to correct the grammar or rewrite the text in a manuscript for an academic journal. For fixing grammar, there is a consensus that it is ethical to use ChatGPT, with 94% of the researchers replying that it is not unethical. However, for rewriting, this becomes more open questions with only 61% respondents thinking that is not unethical. We allow respondents to answer “maybe” to these questions to identify which researchers are uncertain about this question. However, in our analysis, we categorize respondents into those who believe it is ethical and those who either answered that it is unethical or indicate that it is “maybe” unethical. When we compare native English speakers to non-native ones, we find that there is not a significant difference in perceptions of ethics around fixing grammar: 7% compared to 5% (twosample test of proportions yields p=0.211). However, when it comes to rewriting text, 46% of native English speakers think that it is unethical to use ChatGPT, compared to 36% of non-native speakers. A two-sample test of proportions yields p = 0.121 on a two-tailed test or p = 0.060 on the one-tailed. When we compare professors to students and postdocs, we do not find any significant difference for either fixing grammar and rewriting text (for grammar p=0.76 and for rewrite p=0.977). Finally, Table 5 shows the perceptions of respondents regarding whether one should report the use of the ChatGPT for assistance in manuscript preparation. This includes those who consider it ethical to use it and those who believe it to be unethical or are unsure (answered “maybe”). A significant difference exists among these groups. When researchers perceive it as ethical to use ChatGPT, only 21% (30%) believe that reporting is necessary when using it for grammar correction (rewriting). In contrast, researchers who view it as unethical or are uncertain about its ethics have a higher percentage, with 47% (87%) believing that reporting is necessary. This difference among the groups is significant. A two-sample test of proportions yields p=0.017 (0.0001). This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals A Survey Screenshots B Additional Tables Table 3 provides summary statistics for whether authors should acknowledge different forms of assistance. The top panel provides the average responses broken out by being a native English speaker or not, and the bottom panel by being a professor or not. Table 4 shows the results of asking if it is unethical to use ChatGPT to correct the grammar or rewrite the text in a manuscript for an academic journal. For fixing grammar, there is a consensus that it is ethical to use ChatGPT, with 94% of the researchers replying that it is not unethical. However, for rewriting, this becomes more open questions with only 61% respondents thinking that is not unethical. We allow respondents to answer “maybe” to these questions to identify which researchers are uncertain about this question. However, in our analysis, we categorize respondents into those who believe it is ethical and those who either answered that it is unethical or indicate that it is “maybe” unethical. When we compare native English speakers to non-native ones, we find that there is not a significant difference in perceptions of ethics around fixing grammar: 7% compared to 5% (twosample test of proportions yields p=0.211). However, when it comes to rewriting text, 46% of native English speakers think that it is unethical to use ChatGPT, compared to 36% of non-native speakers. A two-sample test of proportions yields p = 0.121 on a two-tailed test or p = 0.060 on the one-tailed. When we compare professors to students and postdocs, we do not find any significant difference for either fixing grammar and rewriting text (for grammar p=0.76 and for rewrite p=0.977). Finally, Table 5 shows the perceptions of respondents regarding whether one should report the use of the ChatGPT for assistance in manuscript preparation. This includes those who consider it ethical to use it and those who believe it to be unethical or are unsure (answered “maybe”). A significant difference exists among these groups. When researchers perceive it as ethical to use ChatGPT, only 21% (30%) believe that reporting is necessary when using it for grammar correction (rewriting). In contrast, researchers who view it as unethical or are uncertain about its ethics have a higher percentage, with 47% (87%) believing that reporting is necessary. This difference among the groups is significant. A two-sample test of proportions yields p=0.017 (0.0001). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

AI Use in Manuscript Preparation for Academic Journals: Appendix for Perceptions and Detection 

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 3 Results 3.1 Reporting Views and Detection Evaluations Figure 1 presents the fraction of survey respondents who indicated that using ChatGPT to fix grammar or rewrite text should be acknowledged. We find substantial differences in reporting views between using ChatGPT for fixing grammar and using it to rewrite text, with 22% of the respondents indicating that grammar correction should be reported relative to 52% for text rewriting (a two-sided test of proportions gives p<0.0001). These perceptions were cleanly nested, as 95% of respondents who thought grammar should be reported also thought rewriting should be reported (only 3 respondents thought that fixing grammar should be reported but not rewriting.) On aggregate, survey respondents viewed these types of AI use differently, but did the AI detector we study treat them differently? Figure 2 shows that the distribution of AI scores is skewed more to the right for abstracts revised using the Rewrite 1 prompt than for those revised using the Grammar 1 prompt. However, both Grammar 1 and Rewrite 1 produce a large number of high AI scores, and if we look again in Figure 1, the 75th percentile values of the AI scores for the abstracts produced by these two prompts are both near the maximal value. It is worth noting that while the detector gave abstracts revised using both prompts high scores, gave much lower scores to the original abstracts. Thus, it had a high degree of accuracy in separating manuscripts that were not revised by ChatGPT from those that were revised in some way. However, this analysis does not indicate how Grammar 1 and Rewrite 1 compare for a given abstract. It could be that abstracts always had higher AI scores when the Rewrite 1 prompt was used. Looking at the abstract level, 2.2% of abstracts had the same AI score for both types of prompts and 24.2% of abstracts had an higher AI score when the Grammar 1 prompt was used than when the Rewrite 1 prompt was used. These results combine to produce our first main finding: Main Finding 3.1 The academics we surveyed were less likely to think that using AI to fix the grammar in manuscripts should be reported than using AI to rewrite manuscripts, but detection software did not always draw this distinction, as abstracts for which GPT-3.5 was used to fix grammar were often flagged as having a high chance of being written by AI. 3.2 Heterogeneity of Perceptions When we compare our survey results for different sources of assistance, we find very similar reporting preferences between ChatGPT and the help of a research assistant (RA) for grammar correction or for rewriting text, which is illustrated in Figure 2. More detailed summary statistics are provided in Appendix B. Comparing the difference between RA and ChatGPT for fixing grammar and rewriting text, we find no significant difference (p=0.5418 for fixing grammar and p=0.6062 for rewriting for two-sided tests of proportions). However, respondents indicated that other tools used for fixing grammar, such as Grammarly and Word, should be acknowledged at even lower rates than ChatGPT, even though these tools might provide users with similar grammar corrections as ChatGPT. Only 14% (5%) of those completing the survey responded that researchers should report using Grammarly (Word) to fix grammar for academic text. A two-sided test of proportions between reporting ChatGPT and Grammarly (Word) for fixing grammar gives p=0.0100 (p<0.0001). This might indicate that researchers are unfamiliar with the differences between these tools, especially given that Grammarly uses some AI models (Fitria 2021). Alternatively, respondents may be so familiar with those tools that they are less worried about their influence on academic writing compared to ChatGPT, which is relatively new. Finally, only 27% of those completing the survey responded that proofreading services should be acknowledged for rewriting academic text, which is much lower than for ChatGPT (52%) and for RA help (49.6%). A two-sided test of proportions comparing ChatGPT to proofreading for rewriting text gives p<0.0001. One explanation for this result could be that proofreading might be considered to be closer to fixing grammar than rewriting text (even that we explicitly stated that proofreading would be used for rewriting text). Another potential explanation could stem from an academic norm related to acknowledging this service or the fact that this service is paid. This leads to our second main finding: Main Finding 3.2 We found little difference in preferences for reporting ChatGPT and RA help, but significant differences in reporting preferences between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Next, we investigate the potential reasons why some academics we surveyed thought that using ChatGPT to rewrite text should be reported and why others did not. Figure 4 shows how reporting preferences differ by English language background (native speaker or not), academic role (professor or not), and perceptions of ethics (whether using ChatGPT to rewrite text is unethical or not). To increase statistical power, we collapse role into professor or not and pool together those who answered either “yes” or “maybe” to the survey question on the ethics of using ChatGPT to rewrite text. When we compare native English speakers to non-native ones, we find that on average native speakers are more inclined towards reporting the use of ChatGPT for rewriting text. When we compare professors (both tenured and untenured) to students and postdocs we find that, on average, postdocs and students are more likely to believe that authors should acknowledge the use of ChatGPT when rewriting text. One possible explanation is that early career researchers are more conservative because they are unfamiliar with the norms in the profession and prefer to take the safer option and report. On the other hand, they may have a better sense of the power of these tools, and hence might feel that reporting is more necessary. Finally, we find large differences in reporting preferences based on perceptions of ethics. Survey respondents who believe that it is unethical to use ChatGPT to rewrite text are almost three times more likely to believe that it should be reported. A more detailed analysis of perceptions of ethics is provided in Appendix B. When we run an OLS regression analysis with only English language background (dummy variable Native=1) and academic role (dummy variable Professor=1) as the explanatory variables, as presented in first column of Table 2, we find that the coefficients are significant at the 5% and 10% level for Native and Professor respectively. However, when we add perceptions of ethics to the regression, the effect for Native becomes weaker and less significant. This suggests that some of this effect was due to differences in perceptions of ethics between native and non-native speakers of English. Main Finding 3.3 We found disagreements among the academics we surveyed about whether using Chat-GPT to rewrite text needs to be reported, and differences were related to perceptions of ethics, academic role, and English language background. 3.3 Detection Robustness First, we compare the 75th percentile of AI scores across all of the prompts provided in Table 1, and Figure 5 shows that we find no perceptible differences for Grammar 2 and Rewrite 2. However, we find that dropping the requirement that the GPT-3.5 output be only one paragraph dramatically reduces the 75th percentile value when revising grammar (Grammar 1b). However, the result is more robust if we consider the 90th percentile instead, as shown in Figure 6. Second, we checked if the 75th percentile values of the AI scores were different in the years before and after the launch of ChatGPT. Looking at the 75th percentile of AI scores in Figure 7, we find a very slight decrease in the 75th percentile AI scores for original abstracts that appeared in 2023. Given this small change for original abstracts, it might not be surprisingly that we do not see much of a difference for Grammar 1 or Rewrite 1. Third, we ran 1,016 of our abstracts through ChatGPT twice to test for the variability in AI scores due to any stochasticity in ChatGPT. Figure 8 shows that there is very little difference in the 75th percentile of scores. This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 3 Results 3.1 Reporting Views and Detection Evaluations Figure 1 presents the fraction of survey respondents who indicated that using ChatGPT to fix grammar or rewrite text should be acknowledged. We find substantial differences in reporting views between using ChatGPT for fixing grammar and using it to rewrite text, with 22% of the respondents indicating that grammar correction should be reported relative to 52% for text rewriting (a two-sided test of proportions gives p<0.0001). These perceptions were cleanly nested, as 95% of respondents who thought grammar should be reported also thought rewriting should be reported (only 3 respondents thought that fixing grammar should be reported but not rewriting.) On aggregate, survey respondents viewed these types of AI use differently, but did the AI detector we study treat them differently? Figure 2 shows that the distribution of AI scores is skewed more to the right for abstracts revised using the Rewrite 1 prompt than for those revised using the Grammar 1 prompt. However, both Grammar 1 and Rewrite 1 produce a large number of high AI scores, and if we look again in Figure 1, the 75th percentile values of the AI scores for the abstracts produced by these two prompts are both near the maximal value. It is worth noting that while the detector gave abstracts revised using both prompts high scores, gave much lower scores to the original abstracts. Thus, it had a high degree of accuracy in separating manuscripts that were not revised by ChatGPT from those that were revised in some way. However, this analysis does not indicate how Grammar 1 and Rewrite 1 compare for a given abstract. It could be that abstracts always had higher AI scores when the Rewrite 1 prompt was used. Looking at the abstract level, 2.2% of abstracts had the same AI score for both types of prompts and 24.2% of abstracts had an higher AI score when the Grammar 1 prompt was used than when the Rewrite 1 prompt was used. These results combine to produce our first main finding: Main Finding 3.1 The academics we surveyed were less likely to think that using AI to fix the grammar in manuscripts should be reported than using AI to rewrite manuscripts, but detection software did not always draw this distinction, as abstracts for which GPT-3.5 was used to fix grammar were often flagged as having a high chance of being written by AI. Main Finding 3.1 The academics we surveyed were less likely to think that using AI to fix the grammar in manuscripts should be reported than using AI to rewrite manuscripts, but detection software did not always draw this distinction, as abstracts for which GPT-3.5 was used to fix grammar were often flagged as having a high chance of being written by AI. 3.2 Heterogeneity of Perceptions When we compare our survey results for different sources of assistance, we find very similar reporting preferences between ChatGPT and the help of a research assistant (RA) for grammar correction or for rewriting text, which is illustrated in Figure 2. More detailed summary statistics are provided in Appendix B. Comparing the difference between RA and ChatGPT for fixing grammar and rewriting text, we find no significant difference (p=0.5418 for fixing grammar and p=0.6062 for rewriting for two-sided tests of proportions). However, respondents indicated that other tools used for fixing grammar, such as Grammarly and Word, should be acknowledged at even lower rates than ChatGPT, even though these tools might provide users with similar grammar corrections as ChatGPT. Only 14% (5%) of those completing the survey responded that researchers should report using Grammarly (Word) to fix grammar for academic text. A two-sided test of proportions between reporting ChatGPT and Grammarly (Word) for fixing grammar gives p=0.0100 (p<0.0001). This might indicate that researchers are unfamiliar with the differences between these tools, especially given that Grammarly uses some AI models (Fitria 2021). Alternatively, respondents may be so familiar with those tools that they are less worried about their influence on academic writing compared to ChatGPT, which is relatively new. Finally, only 27% of those completing the survey responded that proofreading services should be acknowledged for rewriting academic text, which is much lower than for ChatGPT (52%) and for RA help (49.6%). A two-sided test of proportions comparing ChatGPT to proofreading for rewriting text gives p<0.0001. One explanation for this result could be that proofreading might be considered to be closer to fixing grammar than rewriting text (even that we explicitly stated that proofreading would be used for rewriting text). Another potential explanation could stem from an academic norm related to acknowledging this service or the fact that this service is paid. This leads to our second main finding: Main Finding 3.2 We found little difference in preferences for reporting ChatGPT and RA help, but significant differences in reporting preferences between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Main Finding 3.2 We found little difference in preferences for reporting ChatGPT and RA help, but significant differences in reporting preferences between these sources of assistance and paid proofreading and other AI assistant tools (Grammarly and Word). Next, we investigate the potential reasons why some academics we surveyed thought that using ChatGPT to rewrite text should be reported and why others did not. Figure 4 shows how reporting preferences differ by English language background (native speaker or not), academic role (professor or not), and perceptions of ethics (whether using ChatGPT to rewrite text is unethical or not). To increase statistical power, we collapse role into professor or not and pool together those who answered either “yes” or “maybe” to the survey question on the ethics of using ChatGPT to rewrite text. When we compare native English speakers to non-native ones, we find that on average native speakers are more inclined towards reporting the use of ChatGPT for rewriting text. When we compare professors (both tenured and untenured) to students and postdocs we find that, on average, postdocs and students are more likely to believe that authors should acknowledge the use of ChatGPT when rewriting text. One possible explanation is that early career researchers are more conservative because they are unfamiliar with the norms in the profession and prefer to take the safer option and report. On the other hand, they may have a better sense of the power of these tools, and hence might feel that reporting is more necessary. Finally, we find large differences in reporting preferences based on perceptions of ethics. Survey respondents who believe that it is unethical to use ChatGPT to rewrite text are almost three times more likely to believe that it should be reported. A more detailed analysis of perceptions of ethics is provided in Appendix B. When we run an OLS regression analysis with only English language background (dummy variable Native=1) and academic role (dummy variable Professor=1) as the explanatory variables, as presented in first column of Table 2, we find that the coefficients are significant at the 5% and 10% level for Native and Professor respectively. However, when we add perceptions of ethics to the regression, the effect for Native becomes weaker and less significant. This suggests that some of this effect was due to differences in perceptions of ethics between native and non-native speakers of English. Main Finding 3.3 We found disagreements among the academics we surveyed about whether using Chat-GPT to rewrite text needs to be reported, and differences were related to perceptions of ethics, academic role, and English language background. Main Finding 3.3 We found disagreements among the academics we surveyed about whether using Chat-GPT to rewrite text needs to be reported, and differences were related to perceptions of ethics, academic role, and English language background. 3.3 Detection Robustness First, we compare the 75th percentile of AI scores across all of the prompts provided in Table 1, and Figure 5 shows that we find no perceptible differences for Grammar 2 and Rewrite 2. However, we find that dropping the requirement that the GPT-3.5 output be only one paragraph dramatically reduces the 75th percentile value when revising grammar (Grammar 1b). However, the result is more robust if we consider the 90th percentile instead, as shown in Figure 6. Second, we checked if the 75th percentile values of the AI scores were different in the years before and after the launch of ChatGPT. Looking at the 75th percentile of AI scores in Figure 7, we find a very slight decrease in the 75th percentile AI scores for original abstracts that appeared in 2023. Given this small change for original abstracts, it might not be surprisingly that we do not see much of a difference for Grammar 1 or Rewrite 1. Third, we ran 1,016 of our abstracts through ChatGPT twice to test for the variability in AI scores due to any stochasticity in ChatGPT. Figure 8 shows that there is very little difference in the 75th percentile of scores. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

AI Use in Manuscript Preparation for Academic Journals: Results

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 4 Discussion Our findings are a starting point for future research and suggest that several issues need to be carefully considered by the academic community. For instance, the community needs to decide which forms of assistance should be reported, whether it be ChatGPT or some other source, such as RA help or help from another AI tool. Additionally, it needs to decide which types of assistance to be reported, be it fixing grammar or something more extensive. We found more consensus that using ChatGPT to fix grammar does not need to be reported than when it comes to rewriting text. One avenue could be to disclose the actual prompts that are used to revise the paper. Along these lines, Grammarly has a new feature that allows users to acknowledge the usage of AI and the actual prompts that the users used. In addition, our research raises the question of how journals, conferences, and associations can enforce differences in what should be reported. While the detection tool that we employed was able to detect relatively accurately whether AI was used at all, abstracts that were rewritten by GPT-3.5 were sometimes given a lower chance of being written by AI than the grammar-fixed abstracts. This also opens the question of whether using ChatGPT to fix grammar might inadvertently make more substantial changes than desired. How can researchers be sure that they use AI in the desired way? One solution could be to use ChatGPT to point out grammar errors, but to fix them manually, so that ChatGPT does not actually revise the text. These tools and our perceptions of them will surely evolve, but the aim of this paper is to determine how they are perceived and detected in this moment in time, as it appears to be an inflection point in AI ability and in its use to revise text. 4.1 Limitations and Future Directions In terms of our survey design, the use of a convenience sample may have introduced selection issues that complicate our comparisons by role and English language background. In addition, our convenience sample was largely composed of economists, and because views might differ across fields, it would be valuable to also consider what perceptions look like more generally and to illuminate differences across fields. An alternative approach that allows for an assessment of perceptions across fields is offered by Bringula (2023), who runs a sentiment analysis on papers written about AI use in manuscript preparation and finds that the sentiment in those papers is generally positive. Also, the use of a convenience sample did not lend itself to randomizing on the form of assistance (ChatGPT, RA assistance, etc.). In addition, another limitation of our survey is that we do not dig deeply into the nature of ethical perceptions. Since these perceptions were such an important predictor of reporting preferences, it might be insightful to know why academics feel that using AI tools for manuscript preparation is unethical. For example, is someone harmed by their use – such as other academics, science in general, or the authors whose material is used in training the AI – or is there a deeper moral question at play? One way to tease apart these subtleties would be to have academics evaluate a number of detailed vignettes. It might also be interesting to determine the role of payment in ethical considerations, as ChatGPT, RAs, and proofreaders sometimes require payment and sometimes do not. In terms of our detection design, we just considered published papers, at a top journal, and for the field of management. To understand whether these results hold more generally, it would be necessary to look at papers published in other fields, perhaps using a service like Scopus, and to look at working papers, perhaps sourced from SSRN or arXiv. In addition, it might be valuable to consider full papers instead of just abstracts, or as a middle case between abstracts and full papers, introductions might also be useful to examine. Additionally, our results are limited to one AI detection service, so we do not know if they extend to other services, such as GPTZero. Also, it might be of interest to see if other AI-based revision services, such as Grammarly, are flagged by AI detectors too. Along these lines, it might be insightful to consider other forms of writing assistance that might be taken besides fixing grammar and rewriting text, or specific forms of rewriting text. One important dimension could be whether use of chat-based and completion-based AI tools lead to different detection rates. Finally, and related to the last point, it would be valuable to consider a range of different prompts, especially given the sensitivity that we found to relatively small changes in prompt language. One systematic way to choose the prompts could be to hold focus groups of researchers or ChatGPT users or by having a way for researchers to vote on the prompts that are tested. Another dimension of investigation could be to see if our results are sensitive to other important features of the GPT API inputs, such as increasing the model to GPT-4, increasing the temperature to increase the hallucination rate, or by using other system prompts. Given the black box nature of LLMs, a robust empirical analysis is needed to inform policymakers at associations, journals, and conferences of the link between how AI is used and how it is flagged by detection software. This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 4 Discussion Our findings are a starting point for future research and suggest that several issues need to be carefully considered by the academic community. For instance, the community needs to decide which forms of assistance should be reported, whether it be ChatGPT or some other source, such as RA help or help from another AI tool. Additionally, it needs to decide which types of assistance to be reported, be it fixing grammar or something more extensive. We found more consensus that using ChatGPT to fix grammar does not need to be reported than when it comes to rewriting text. One avenue could be to disclose the actual prompts that are used to revise the paper. Along these lines, Grammarly has a new feature that allows users to acknowledge the usage of AI and the actual prompts that the users used. In addition, our research raises the question of how journals, conferences, and associations can enforce differences in what should be reported. While the detection tool that we employed was able to detect relatively accurately whether AI was used at all, abstracts that were rewritten by GPT-3.5 were sometimes given a lower chance of being written by AI than the grammar-fixed abstracts. This also opens the question of whether using ChatGPT to fix grammar might inadvertently make more substantial changes than desired. How can researchers be sure that they use AI in the desired way? One solution could be to use ChatGPT to point out grammar errors, but to fix them manually, so that ChatGPT does not actually revise the text. These tools and our perceptions of them will surely evolve, but the aim of this paper is to determine how they are perceived and detected in this moment in time, as it appears to be an inflection point in AI ability and in its use to revise text. 4.1 Limitations and Future Directions In terms of our survey design, the use of a convenience sample may have introduced selection issues that complicate our comparisons by role and English language background. In addition, our convenience sample was largely composed of economists, and because views might differ across fields, it would be valuable to also consider what perceptions look like more generally and to illuminate differences across fields. An alternative approach that allows for an assessment of perceptions across fields is offered by Bringula (2023), who runs a sentiment analysis on papers written about AI use in manuscript preparation and finds that the sentiment in those papers is generally positive. Also, the use of a convenience sample did not lend itself to randomizing on the form of assistance (ChatGPT, RA assistance, etc.). In addition, another limitation of our survey is that we do not dig deeply into the nature of ethical perceptions. Since these perceptions were such an important predictor of reporting preferences, it might be insightful to know why academics feel that using AI tools for manuscript preparation is unethical. For example, is someone harmed by their use – such as other academics, science in general, or the authors whose material is used in training the AI – or is there a deeper moral question at play? One way to tease apart these subtleties would be to have academics evaluate a number of detailed vignettes. It might also be interesting to determine the role of payment in ethical considerations, as ChatGPT, RAs, and proofreaders sometimes require payment and sometimes do not. In terms of our detection design, we just considered published papers, at a top journal, and for the field of management. To understand whether these results hold more generally, it would be necessary to look at papers published in other fields, perhaps using a service like Scopus, and to look at working papers, perhaps sourced from SSRN or arXiv. In addition, it might be valuable to consider full papers instead of just abstracts, or as a middle case between abstracts and full papers, introductions might also be useful to examine. Additionally, our results are limited to one AI detection service, so we do not know if they extend to other services, such as GPTZero. Also, it might be of interest to see if other AI-based revision services, such as Grammarly, are flagged by AI detectors too. Along these lines, it might be insightful to consider other forms of writing assistance that might be taken besides fixing grammar and rewriting text, or specific forms of rewriting text. One important dimension could be whether use of chat-based and completion-based AI tools lead to different detection rates. Finally, and related to the last point, it would be valuable to consider a range of different prompts, especially given the sensitivity that we found to relatively small changes in prompt language. One systematic way to choose the prompts could be to hold focus groups of researchers or ChatGPT users or by having a way for researchers to vote on the prompts that are tested. Another dimension of investigation could be to see if our results are sensitive to other important features of the GPT API inputs, such as increasing the model to GPT-4, increasing the temperature to increase the hallucination rate, or by using other system prompts. Given the black box nature of LLMs, a robust empirical analysis is needed to inform policymakers at associations, journals, and conferences of the link between how AI is used and how it is flagged by detection software. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

AI Use in Manuscript Preparation for Academic Journals: Discussion

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals References Agrawal, Ajay, Joshua Gans, and Avi Goldfarb (2019). The economics of artificial intelligence: an agenda. University of Chicago Press. Akram, Arslan (2023). “An Empirical Study of AI Generated Text Detection Tools”. arXiv preprint arXiv:2310.01423. Altmäe, Signe, Alberto Sola-Leyva, and Andres Salumets (2023). “Artificial intelligence in scientific writing: a friend or a foe?” Reproductive BioMedicine Online Athey, Susan and Guido W Imbens (2019). “Machine learning methods that economists should know about”. Annual Review of Economics 11, pp. 685–725. Beck, J Thaddeus et al. (2020). “Artificial intelligence tool for optimizing eligibility screening for clinical trials in a large community cancer center”. JCO clinical cancer informatics 4, pp. 50–59. Björkegren, Daniel, Joshua E. Blumenstock, and Samsun Knight (2022). (Machine) Learning What Policies Value. arXiv: 2206.00727 [econ.GN]. Bom, Hee-Seung Henry (2023). “Exploring the Opportunities and Challenges of ChatGPT in Academic Writing: a Roundtable Discussion”. Nuclear Medicine and Molecular Imaging, pp. 1– 3. Bommasani, Rishi et al. (2021). “On the opportunities and risks of foundation models”. arXiv preprint arXiv:2108.07258. Bringula, Rex (2023). “What do academics have to say about ChatGPT? A text mining analytics on the discussions regarding ChatGPT on research writing”. AI and Ethics, pp. 1–13. Capra, C. Monica, Matthew Gomies, and Shanshan Zhang (2023). “The Sound of Cooperation and Deception in High Stakes Interactions”. Charness, Gary, Brian Jabarian, and John A List (2023). Generation next: Experimentation with ai. Tech. rep. National Bureau of Economic Research. Chien, Chen-Fu et al. (2020). Artificial intelligence in manufacturing and logistics systems: algorithms, applications, and case studies. Cowen, Tyler and Alexander T Tabarrok (2023). “How to learn and teach economics with large language models, including GPT”. Including GPT (March 17, 2023). Daun, Marian and Jennifer Brings (2023). “How ChatGPT will change software engineering education”. In: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pp. 110–116. Deza, Arturo, Amit Surana, and Miguel P Eckstein (2019). “Assessment of faster r-cnn in manmachine collaborative search”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3185–3194. Dietvorst, Berkeley J, Joseph P Simmons, and Cade Massey (2015). “Algorithm aversion: people erroneously avoid algorithms after seeing them err.” Journal of Experimental Psychology: General 144.1, p. 114. Dranove, David and Ginger Zhe Jin (2010). “Quality disclosure and certification: Theory and practice”. Journal of Economic Literature 48.4, pp. 935–63. Farrell, Max H, Tengyuan Liang, and Sanjog Misra (2020). “Deep learning for individual heterogeneity: An automatic inference framework”. arXiv preprint arXiv:2010.14694. Fitria, Tira Nur (2021). “Grammarly as AI-powered English writing assistant: Students’ alternative for writing English”. Metathesis: Journal of English Language, Literature, and Teaching 5.1, pp. 65–78. Franchi, Matt et al. (2023). “Detecting disparities in police deployments using dashcam data”. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 534–544. Fudenberg, Drew and Annie Liang (2019). “Predicting and understanding initial play”. American Economic Review 109.12, pp. 4112–4141 Fyfe, Paul (2023). “How to cheat on your final paper: Assigning AI for student writing”. AI & SOCIETY 38.4, pp. 1395–1405. Gajos, Krzysztof Z and Lena Mamykina (2022). “Do people engage cognitively with AI? Impact of AI assistance on incidental learning”. In: 27th international conference on intelligent user interfaces, pp. 794–806. Hill-Yardin, Elisa L et al. (2023). “A Chat (GPT) about the future of scientific publishing”. Brain Behav Immun 110, pp. 152–154. Horton, John J (2023). Large language models as simulated economic agents: What can we learn from homo silicus? Tech. rep. National Bureau of Economic Research. Ibrahim, Hazem et al. (2023). “Rethinking Homework in the Age of Artificial Intelligence”. IEEE Intelligent Systems 38.2, pp. 24–27. Jin, Ginger Zhe, Michael Luca, and Daniel Martin (2015). Is no news (perceived as) bad news? An experimental investigation of information disclosure. Tech. rep. National Bureau of Economic Research. Jungherr, Andreas (2023). “Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments”. Korinek, Anton (2023). “Language models and cognitive automation for economic research” Lambrecht, Anja and Catherine Tucker (2019). “Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads”. Management science 65.7, pp. 2966–2981. Malik, Agung Rinaldy et al. (2023). “Exploring Artificial Intelligence in Academic Essay: Higher Education Student’s Perspective”. International Journal of Educational Research Open 5, p. 100296. Mullainathan, Sendhil and Jann Spiess (2017). “Machine learning: an applied econometric approach”. Journal of Economic Perspectives 31.2, pp. 87–106. Obermeyer, Ziad et al. (2019). “Dissecting racial bias in an algorithm used to manage the health of populations”. Science 366.6464, pp. 447–453. Pallathadka, Harikumar et al. (2023). “Applications of artificial intelligence in business management, e-commerce and finance”. Materials Today: Proceedings 80, pp. 2610–2613. Rambachan, Ashesh et al. (2021). “Identifying prediction mistakes in observational data”. Harvard University. Ray, Partha Pratim (2023). “ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope”. Internet of Things and Cyber-Physical Systems. Rolf, Esther et al. (2020). Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning. arXiv: 2003.06740 [cs.LG]. Salah, Mohammed, Hussam Al Halbusi, and Fadi Abdelfattah (2023). “May the force of text data analysis be with you: Unleashing the power of generative AI for social psychology research”. Computers in Human Behavior: Artificial Humans, p. 100006 Schmohl, Tobias et al. (2020). “How Artificial Intelligence can improve the Academic Writing of Students”. In: Conference Proceedings. The Future of Education 2020. Shahriar, Sakib and Kadhim Hayawi (2023). “Let’s have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations”. arXiv preprint arXiv:2302.13817. Singh, Harjit and Avneet Singh (2023). “ChatGPT: Systematic Review, Applications, and Agenda for Multidisciplinary Research”. Journal of Chinese Economic and Business Studies 21.2, pp. 193– 212. Steyvers, Mark et al. (2022). “Bayesian modeling of human–AI complementarity”. Proceedings of the National Academy of Sciences 119.11, e2111547119. Sundar, S Shyam and Eun-Ju Lee (2022). “Rethinking communication in the era of artificial intelligence”. Human Communication Research 48.3, pp. 379–385. Tejeda, Heliodoro et al. (2022). “AI-assisted decision-making: A cognitive modeling approach to infer latent reliance strategies”. Computational Brain & Behavior 5.4, pp. 491–508. Thorp, H Holden (2023). ChatGPT is fun, but not an author. Wang, Xinru, Chen Liang, and Ming Yin (2023). “The Effects of AI Biases and Explanations on Human Decision Fairness: A Case Study of Bidding in Rental Housing Markets”. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, Edith Elkind (Ed.). International Joint Conferences on Artificial Intelligence Organization, pp. 3076– 3084 Yang, Nanyin, Marco Palma, and Andreas Drichoutis (2023). “How Does Humanizing Virtual Assistants Affect the Propensity to Follow Their Advice?” Yang, Nanyin, Marco A Palma, and Andreas C Drichoutis (2023). “How Does Humanizing Virtual Assistants Affect the Propensity to Follow Their Advice?” Zuiderwijk, Anneke, Yu-Che Chen, and Fadi Salem (2021). “Implications of the use of artificial intelligence in public governance: A systematic literature review and a research agenda”. Government Information Quarterly 38.3, p. 101577. This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals References Agrawal, Ajay, Joshua Gans, and Avi Goldfarb (2019). The economics of artificial intelligence: an agenda. University of Chicago Press. Akram, Arslan (2023). “An Empirical Study of AI Generated Text Detection Tools”. arXiv preprint arXiv:2310.01423. Altmäe, Signe, Alberto Sola-Leyva, and Andres Salumets (2023). “Artificial intelligence in scientific writing: a friend or a foe?” Reproductive BioMedicine Online Athey, Susan and Guido W Imbens (2019). “Machine learning methods that economists should know about”. Annual Review of Economics 11, pp. 685–725. Beck, J Thaddeus et al. (2020). “Artificial intelligence tool for optimizing eligibility screening for clinical trials in a large community cancer center”. JCO clinical cancer informatics 4, pp. 50–59. Björkegren, Daniel, Joshua E. Blumenstock, and Samsun Knight (2022). (Machine) Learning What Policies Value. arXiv: 2206.00727 [econ.GN]. Bom, Hee-Seung Henry (2023). “Exploring the Opportunities and Challenges of ChatGPT in Academic Writing: a Roundtable Discussion”. Nuclear Medicine and Molecular Imaging, pp. 1– 3. Bommasani, Rishi et al. (2021). “On the opportunities and risks of foundation models”. arXiv preprint arXiv:2108.07258. Bringula, Rex (2023). “What do academics have to say about ChatGPT? A text mining analytics on the discussions regarding ChatGPT on research writing”. AI and Ethics, pp. 1–13. Capra, C. Monica, Matthew Gomies, and Shanshan Zhang (2023). “The Sound of Cooperation and Deception in High Stakes Interactions”. Charness, Gary, Brian Jabarian, and John A List (2023). Generation next: Experimentation with ai. Tech. rep. National Bureau of Economic Research. Chien, Chen-Fu et al. (2020). Artificial intelligence in manufacturing and logistics systems: algorithms, applications, and case studies. Cowen, Tyler and Alexander T Tabarrok (2023). “How to learn and teach economics with large language models, including GPT”. Including GPT (March 17, 2023). Daun, Marian and Jennifer Brings (2023). “How ChatGPT will change software engineering education”. In: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pp. 110–116. Deza, Arturo, Amit Surana, and Miguel P Eckstein (2019). “Assessment of faster r-cnn in manmachine collaborative search”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3185–3194. Dietvorst, Berkeley J, Joseph P Simmons, and Cade Massey (2015). “Algorithm aversion: people erroneously avoid algorithms after seeing them err.” Journal of Experimental Psychology: General 144.1, p. 114. Dranove, David and Ginger Zhe Jin (2010). “Quality disclosure and certification: Theory and practice”. Journal of Economic Literature 48.4, pp. 935–63. Farrell, Max H, Tengyuan Liang, and Sanjog Misra (2020). “Deep learning for individual heterogeneity: An automatic inference framework”. arXiv preprint arXiv:2010.14694. Fitria, Tira Nur (2021). “Grammarly as AI-powered English writing assistant: Students’ alternative for writing English”. Metathesis: Journal of English Language, Literature, and Teaching 5.1, pp. 65–78. Franchi, Matt et al. (2023). “Detecting disparities in police deployments using dashcam data”. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 534–544. Fudenberg, Drew and Annie Liang (2019). “Predicting and understanding initial play”. American Economic Review 109.12, pp. 4112–4141 Fyfe, Paul (2023). “How to cheat on your final paper: Assigning AI for student writing”. AI & SOCIETY 38.4, pp. 1395–1405. Gajos, Krzysztof Z and Lena Mamykina (2022). “Do people engage cognitively with AI? Impact of AI assistance on incidental learning”. In: 27th international conference on intelligent user interfaces, pp. 794–806. Hill-Yardin, Elisa L et al. (2023). “A Chat (GPT) about the future of scientific publishing”. Brain Behav Immun 110, pp. 152–154. Horton, John J (2023). Large language models as simulated economic agents: What can we learn from homo silicus? Tech. rep. National Bureau of Economic Research. Ibrahim, Hazem et al. (2023). “Rethinking Homework in the Age of Artificial Intelligence”. IEEE Intelligent Systems 38.2, pp. 24–27. Jin, Ginger Zhe, Michael Luca, and Daniel Martin (2015). Is no news (perceived as) bad news? An experimental investigation of information disclosure. Tech. rep. National Bureau of Economic Research. Jungherr, Andreas (2023). “Using ChatGPT and Other Large Language Model (LLM) Applications for Academic Paper Assignments”. Korinek, Anton (2023). “Language models and cognitive automation for economic research” Lambrecht, Anja and Catherine Tucker (2019). “Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads”. Management science 65.7, pp. 2966–2981. Malik, Agung Rinaldy et al. (2023). “Exploring Artificial Intelligence in Academic Essay: Higher Education Student’s Perspective”. International Journal of Educational Research Open 5, p. 100296. Mullainathan, Sendhil and Jann Spiess (2017). “Machine learning: an applied econometric approach”. Journal of Economic Perspectives 31.2, pp. 87–106. Obermeyer, Ziad et al. (2019). “Dissecting racial bias in an algorithm used to manage the health of populations”. Science 366.6464, pp. 447–453. Pallathadka, Harikumar et al. (2023). “Applications of artificial intelligence in business management, e-commerce and finance”. Materials Today: Proceedings 80, pp. 2610–2613. Rambachan, Ashesh et al. (2021). “Identifying prediction mistakes in observational data”. Harvard University. Ray, Partha Pratim (2023). “ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope”. Internet of Things and Cyber-Physical Systems. Rolf, Esther et al. (2020). Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning. arXiv: 2003.06740 [cs.LG]. Salah, Mohammed, Hussam Al Halbusi, and Fadi Abdelfattah (2023). “May the force of text data analysis be with you: Unleashing the power of generative AI for social psychology research”. Computers in Human Behavior: Artificial Humans, p. 100006 Schmohl, Tobias et al. (2020). “How Artificial Intelligence can improve the Academic Writing of Students”. In: Conference Proceedings. The Future of Education 2020. Shahriar, Sakib and Kadhim Hayawi (2023). “Let’s have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations”. arXiv preprint arXiv:2302.13817. Singh, Harjit and Avneet Singh (2023). “ChatGPT: Systematic Review, Applications, and Agenda for Multidisciplinary Research”. Journal of Chinese Economic and Business Studies 21.2, pp. 193– 212. Steyvers, Mark et al. (2022). “Bayesian modeling of human–AI complementarity”. Proceedings of the National Academy of Sciences 119.11, e2111547119. Sundar, S Shyam and Eun-Ju Lee (2022). “Rethinking communication in the era of artificial intelligence”. Human Communication Research 48.3, pp. 379–385. Tejeda, Heliodoro et al. (2022). “AI-assisted decision-making: A cognitive modeling approach to infer latent reliance strategies”. Computational Brain & Behavior 5.4, pp. 491–508. Thorp, H Holden (2023). ChatGPT is fun, but not an author. Wang, Xinru, Chen Liang, and Ming Yin (2023). “The Effects of AI Biases and Explanations on Human Decision Fairness: A Case Study of Bidding in Rental Housing Markets”. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, Edith Elkind (Ed.). International Joint Conferences on Artificial Intelligence Organization, pp. 3076– 3084 Yang, Nanyin, Marco Palma, and Andreas Drichoutis (2023). “How Does Humanizing Virtual Assistants Affect the Propensity to Follow Their Advice?” Yang, Nanyin, Marco A Palma, and Andreas C Drichoutis (2023). “How Does Humanizing Virtual Assistants Affect the Propensity to Follow Their Advice?” Zuiderwijk, Anneke, Yu-Che Chen, and Fadi Salem (2021). “Implications of the use of artificial intelligence in public governance: A systematic literature review and a research agenda”. Government Information Quarterly 38.3, p. 101577. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

AI Use in Manuscript Preparation for Academic Journals: References

Authors:
(1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu);
(2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction
Methods
Results
Discussion
References
Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 2 Methods In order to answer our main research questions, we split our design into two parts. The first part was a survey of the academic community that captured academics’ views on reporting AI use in manuscript preparation. The second part was to test an AI detector’s reaction to AI use in manuscript preparation. Our primary focus was on using AI to fix grammar and rewrite text, which we consider to be important intermediate cases between not using AI at all and using AI to do all of the writing based on limited inputs (e.g., only a title and/or a set of results). We asked academics for their perceptions about these two kinds of AI usage in manuscript preparation and investigated how the detector responded to these different kinds of usage. All of this will be explained in more detail in the following subsections. 2.1 Survey Design Beginning August 22nd 2023 and running until September 20th 2023, we conducted a brief unpaid survey about perceptions of AI use in manuscript preparation. We sent this survey to a convenience sample of academics; specifically, we sent it to three listservs: the UCSB Economic Department listserv (connecting graduate students, professors, and postdocs at the University of California, Santa Barbara); the Economics Science Association (ESA) announcement listserv (ESA is the leading organization for experimental economics); and the Decision Theory (DT) Forum listserv (the DT Forum is a listserv of academics working on decision theory that is run by Itzhak Gilboa). We sent the survey with a few days delay between each group, which allows for a rough measure of the respondents from each source. We had a total of 271 respondents complete the survey: 38 from UCSB, 199 from UCSB and the ESA community, and 34 from UCSB, the ESA community, and the DT Forum (this number does not include 20 individuals who did not specify an academic role). The median time to answer the survey was 1.21 minutes, and survey respondents provided informed consent (as shown in Appendix A). In this survey, we focused on two main aspects related to perceptions of AI use in manuscript preparation. The first aspect is whether authors should acknowledge using ChatGPT to fix grammar or to rewrite text. We asked this over two separate questions, as shown in Figure 10 in Appendix A. These two uses of ChatGPT reflect to many of the use cases of LLMs suggested in articles, websites, and online courses. The second aspect is whether using ChatGPT to modify an academic manuscript is unethical. Again, we split this into two questions – one for grammar and one for rewrite – to allow researchers to address whether ethics also depends on usage. Because we suspected that there might be differences in these perceptions and an academic’s role and language background, we asked respondents if they were a native speaker of English and their current role (postdoc, student, untenured professor, tenured professor, and/or other). Of the 271 respondents who completed our survey, 83 reported being a native English speaker, and we categorized 67 respondents as students, 32 as postdocs, 59 as untenured professors, and 113 as tenured professors. If someone reported multiple roles, we took the “highest” report, ranked in this order. In addition to these questions, we added follow-up questions on a second page of the survey about whether authors should acknowledge other writing services and tools such as Word, Grammarly, proofreading, and the work of research assistants (see Figure 11 in Appendix A). We asked this question only after the ChatGPT questions and on a new page because our main focus was the perception of AI (e.g., ChatGPT) in academic writing, and because we wanted to avoid these followup questions influencing the answers given about ChatGPT. In addition, we did not randomize the order between ChatGPT and other services because the convenience sample already knew this was a survey about ChatGPT, so that would have been in their mind already when completing the survey. However, by asking questions about ChatGPT first, we might have reduced differences in reported perceptions with other writing services, as people might feel the need to report consistently for a given use. To make the questions about other sources comparable the questions about ChatGPT, we used the same question format and split the comparison into two groups: fixing grammar and rewriting text. The tools we included for fixing grammar included Word, Grammarly, and RA help. Importantly, we wrote explicitly in the question that these tools were to be used for fixing grammar. For rewriting text, we used proofreading and RA help. Once more, we explicitly asked whether authors should acknowledge using these tools and services for rewriting text for an academic journal. 2.2 Detection Design Next, we tested how an AI detector would react to using ChatGPT to fix grammar in academic text and to rewrite academic text. We first collected titles and abstracts from 2,716 papers published in the journal Management Science from January 2013 to September 2023. We excluded articles with the following words in their titles, as they did not appear to be original articles: “Erratum,” “Comment on,” “Management Science,” “Reviewers and Guest Associate Editors,” and “Reviewers and Guest Editors.” We intentionally included papers that were published before the launch of ChatGPT in November 30, 2022 in order to have source text that was plausibly unimpacted by LLM use. We revised these abstracts using a variety of prompts, as seen in Table 1 below. Rather than selecting the prompts ourselves, we wanted to use an external source. We decided to use prompts from the online GitHub page ChatGPT Prompts for Academic Writing, which advises researchers on how to use ChatGPT. We chose this source because it was the first link returned from a Google search of “ChatGPT Prompts for Academic Writing.” Our baselines prompts were “Grammar 1” and “Rewrite 1,” and for robustness we also considered two functionally related prompts, “Grammar 2” and “Rewrite 2.” We added a restriction at the end of these prompts to “Give a version in one paragraph based on this paragraph” in order to meet the requirement that the abstract be only one paragraph in length. To consider the robustness of our results to this additional language, we also ran versions of our baseline prompts that did not include it: “Grammar 1b” and “Rewrite 1b.” It has been shown that there are ways to fool certain detectors, such as by adding strange characters to the text, and this is a constantly evolving game of cat-and-mouse as detectors evolve. Although this could be of interest for researchers, we do not study this particular phenomenon. Instead, it is our goal to see whether an academic who uses ChatGPT for widely-proposed purposes – without further edits or detection avoidance strategies – would have their writing be flagged as AI generated. To revise these abstracts at scale, we leveraged the GPT API with settings that would produce output that closely mirrors the output of a researcher using ChatGPT to revise academic writing. The specific model we used was gpt-3.5-turbo-0613 (GPT-3.5 Turbo released in June 13 2023), and we kept the default settings for temperature, top p, frequency penalty, and presence penalty. We also used the default system prompt, “You are a helpful assistant,” which is a background prompt that can be changed in the API but not in ChatGPT itself. We did a fresh call each time we used a prompt to avoid learning and history effects. Finally, we used a leading paid service (Originality.ai) to see how AI detection algorithms might react to this use of LLMs. We first evaluated the original Management Science abstracts, and then the abstracts revised by GPT-3.5 based on all of the prompts. Originality.ai provides an “AI score,” which is a value between 0% and 100% that is interpreted as the likelihood that AI wrote the text being evaluated. A high score means a high likelihood that AI generated the text. Specifically, the company states: “If an article has an AI score of 5%... there is a 95% chance that the article was human-generated (NOT that 5% of the article is AI generated).” Akram (2023) studied a number of popular detection tools and found that Originality.ai had the highest accuracy rate (97%). This paper is available on arxiv under CC 4.0 license. Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Authors: Authors: (1) Nir Chemaya, University of California, Santa Barbara and (e-mail: nir@ucsb.edu); (2) Daniel Martin, University of California, Santa Barbara and Kellogg School of Management, Northwestern University and (e-mail: danielmartin@ucsb.edu). Table of Links Abstract and Introduction Methods Results Discussion References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Abstract and Introduction Abstract and Introduction Methods Methods Results Results Discussion Discussion References References Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals Appendix for Perceptions and Detection of AI Use in Manuscript Preparation for Academic Journals 2 Methods In order to answer our main research questions, we split our design into two parts. The first part was a survey of the academic community that captured academics’ views on reporting AI use in manuscript preparation. The second part was to test an AI detector’s reaction to AI use in manuscript preparation. Our primary focus was on using AI to fix grammar and rewrite text, which we consider to be important intermediate cases between not using AI at all and using AI to do all of the writing based on limited inputs (e.g., only a title and/or a set of results). We asked academics for their perceptions about these two kinds of AI usage in manuscript preparation and investigated how the detector responded to these different kinds of usage. All of this will be explained in more detail in the following subsections. 2.1 Survey Design Beginning August 22nd 2023 and running until September 20th 2023, we conducted a brief unpaid survey about perceptions of AI use in manuscript preparation. We sent this survey to a convenience sample of academics; specifically, we sent it to three listservs: the UCSB Economic Department listserv (connecting graduate students, professors, and postdocs at the University of California, Santa Barbara); the Economics Science Association (ESA) announcement listserv (ESA is the leading organization for experimental economics); and the Decision Theory (DT) Forum listserv (the DT Forum is a listserv of academics working on decision theory that is run by Itzhak Gilboa). We sent the survey with a few days delay between each group, which allows for a rough measure of the respondents from each source. We had a total of 271 respondents complete the survey: 38 from UCSB, 199 from UCSB and the ESA community, and 34 from UCSB, the ESA community, and the DT Forum (this number does not include 20 individuals who did not specify an academic role). The median time to answer the survey was 1.21 minutes, and survey respondents provided informed consent (as shown in Appendix A). In this survey, we focused on two main aspects related to perceptions of AI use in manuscript preparation. The first aspect is whether authors should acknowledge using ChatGPT to fix grammar or to rewrite text. We asked this over two separate questions, as shown in Figure 10 in Appendix A. These two uses of ChatGPT reflect to many of the use cases of LLMs suggested in articles, websites, and online courses. The second aspect is whether using ChatGPT to modify an academic manuscript is unethical. Again, we split this into two questions – one for grammar and one for rewrite – to allow researchers to address whether ethics also depends on usage. Because we suspected that there might be differences in these perceptions and an academic’s role and language background, we asked respondents if they were a native speaker of English and their current role (postdoc, student, untenured professor, tenured professor, and/or other). Of the 271 respondents who completed our survey, 83 reported being a native English speaker, and we categorized 67 respondents as students, 32 as postdocs, 59 as untenured professors, and 113 as tenured professors. If someone reported multiple roles, we took the “highest” report, ranked in this order. In addition to these questions, we added follow-up questions on a second page of the survey about whether authors should acknowledge other writing services and tools such as Word, Grammarly, proofreading, and the work of research assistants (see Figure 11 in Appendix A). We asked this question only after the ChatGPT questions and on a new page because our main focus was the perception of AI (e.g., ChatGPT) in academic writing, and because we wanted to avoid these followup questions influencing the answers given about ChatGPT. In addition, we did not randomize the order between ChatGPT and other services because the convenience sample already knew this was a survey about ChatGPT, so that would have been in their mind already when completing the survey. However, by asking questions about ChatGPT first, we might have reduced differences in reported perceptions with other writing services, as people might feel the need to report consistently for a given use. To make the questions about other sources comparable the questions about ChatGPT, we used the same question format and split the comparison into two groups: fixing grammar and rewriting text. The tools we included for fixing grammar included Word, Grammarly, and RA help. Importantly, we wrote explicitly in the question that these tools were to be used for fixing grammar. For rewriting text, we used proofreading and RA help. Once more, we explicitly asked whether authors should acknowledge using these tools and services for rewriting text for an academic journal. 2.2 Detection Design Next, we tested how an AI detector would react to using ChatGPT to fix grammar in academic text and to rewrite academic text. We first collected titles and abstracts from 2,716 papers published in the journal Management Science from January 2013 to September 2023. We excluded articles with the following words in their titles, as they did not appear to be original articles: “Erratum,” “Comment on,” “Management Science,” “Reviewers and Guest Associate Editors,” and “Reviewers and Guest Editors.” We intentionally included papers that were published before the launch of ChatGPT in November 30, 2022 in order to have source text that was plausibly unimpacted by LLM use. We revised these abstracts using a variety of prompts, as seen in Table 1 below. Rather than selecting the prompts ourselves, we wanted to use an external source. We decided to use prompts from the online GitHub page ChatGPT Prompts for Academic Writing, which advises researchers on how to use ChatGPT. We chose this source because it was the first link returned from a Google search of “ChatGPT Prompts for Academic Writing.” Our baselines prompts were “Grammar 1” and “Rewrite 1,” and for robustness we also considered two functionally related prompts, “Grammar 2” and “Rewrite 2.” We added a restriction at the end of these prompts to “Give a version in one paragraph based on this paragraph” in order to meet the requirement that the abstract be only one paragraph in length. To consider the robustness of our results to this additional language, we also ran versions of our baseline prompts that did not include it: “Grammar 1b” and “Rewrite 1b.” It has been shown that there are ways to fool certain detectors, such as by adding strange characters to the text, and this is a constantly evolving game of cat-and-mouse as detectors evolve. Although this could be of interest for researchers, we do not study this particular phenomenon. Instead, it is our goal to see whether an academic who uses ChatGPT for widely-proposed purposes – without further edits or detection avoidance strategies – would have their writing be flagged as AI generated. To revise these abstracts at scale, we leveraged the GPT API with settings that would produce output that closely mirrors the output of a researcher using ChatGPT to revise academic writing. The specific model we used was gpt-3.5-turbo-0613 (GPT-3.5 Turbo released in June 13 2023), and we kept the default settings for temperature, top p, frequency penalty, and presence penalty. We also used the default system prompt, “You are a helpful assistant,” which is a background prompt that can be changed in the API but not in ChatGPT itself. We did a fresh call each time we used a prompt to avoid learning and history effects. Finally, we used a leading paid service (Originality.ai) to see how AI detection algorithms might react to this use of LLMs. We first evaluated the original Management Science abstracts, and then the abstracts revised by GPT-3.5 based on all of the prompts. Originality.ai provides an “AI score,” which is a value between 0% and 100% that is interpreted as the likelihood that AI wrote the text being evaluated. A high score means a high likelihood that AI generated the text. Specifically, the company states: “If an article has an AI score of 5%... there is a 95% chance that the article was human-generated (NOT that 5% of the article is AI generated).” Akram (2023) studied a number of popular detection tools and found that Originality.ai had the highest accuracy rate (97%). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Stories, News & Mentions about ACADEMYXI

Academyxi’s Stories on HackerNoon

Academyxi's News & Mentions