WEBVTT 00:00:09.464 --> 00:00:09.945 Hey there, 00:00:10.006 --> 00:00:11.868 we're really pleased to be launching our 00:00:11.948 --> 00:00:13.830 new AI Adoption Path, 00:00:13.849 --> 00:00:16.393 this resource critical for VET providers 00:00:16.452 --> 00:00:18.335 to be looking at responsible 00:00:18.434 --> 00:00:20.797 implementation and adoption of AI 00:00:20.838 --> 00:00:23.882 solutions across all areas of their 00:00:23.981 --> 00:00:24.623 operations. 00:00:25.003 --> 00:00:26.943 This is coming at a critical time for 00:00:26.963 --> 00:00:27.664 the first time. 00:00:27.704 --> 00:00:29.943 We've got definitive guidance from the 00:00:29.963 --> 00:00:32.085 Australian government on the expectations 00:00:32.125 --> 00:00:33.524 for what needs to be in place for 00:00:33.685 --> 00:00:35.965 adoption across organizations and 00:00:36.005 --> 00:00:36.566 businesses. 00:00:37.066 --> 00:00:39.246 And we've designed this pack specifically 00:00:39.286 --> 00:00:42.307 to that guidance and customized to support 00:00:42.326 --> 00:00:43.847 that providers for effective 00:00:44.027 --> 00:00:44.826 implementation. 00:00:45.508 --> 00:00:46.868 Before we get too far into it, 00:00:46.887 --> 00:00:48.448 let's just start with a reflection. 00:00:49.128 --> 00:00:51.088 What are our use cases 00:00:52.128 --> 00:00:55.112 Who is accountable for each one and what 00:00:55.212 --> 00:00:58.755 decision rights do they actually have when 00:00:58.774 --> 00:01:01.057 you're thinking about AI implementation? 00:01:01.737 --> 00:01:05.301 What harm could realistically occur in our 00:01:05.361 --> 00:01:07.022 context to whom? 00:01:08.081 --> 00:01:10.942 And where is our stop or pause threshold 00:01:11.183 --> 00:01:12.424 in the implementations? 00:01:12.963 --> 00:01:15.926 What controls do we have for data testing 00:01:16.126 --> 00:01:18.487 and ongoing monitoring their 00:01:18.528 --> 00:01:19.429 self-assurance? 00:01:19.968 --> 00:01:22.790 So problems are found early rather than 00:01:23.150 --> 00:01:24.871 after an impact occurs. 00:01:25.352 --> 00:01:27.554 Some key questions there that are really 00:01:27.593 --> 00:01:30.756 getting at the responsible implementation 00:01:30.796 --> 00:01:32.837 and use of AI as we adopt 00:01:33.177 --> 00:01:35.739 these AI solutions across our operations. 00:01:36.219 --> 00:01:37.400 What we're going to be doing in this 00:01:37.441 --> 00:01:39.781 short video is exploring the resources 00:01:39.822 --> 00:01:41.063 that are available, 00:01:41.102 --> 00:01:42.603 the tools and templates that we've put 00:01:42.644 --> 00:01:45.585 together for that responsible AI adoption 00:01:45.865 --> 00:01:48.808 with a specific focus on their providers. 00:01:49.128 --> 00:01:50.448 Before I get into the pack, 00:01:50.608 --> 00:01:52.349 I just want to cover off on some 00:01:52.551 --> 00:01:54.212 of those key obligations. 00:01:54.731 --> 00:01:55.191 And yes, 00:01:55.272 --> 00:01:56.673 we do have the twenty twenty five 00:01:56.712 --> 00:01:58.274 standards for RTOs that we need to be 00:01:58.293 --> 00:01:58.834 thinking about. 00:01:59.415 --> 00:02:01.135 And it does come into multiple areas. 00:02:01.176 --> 00:02:02.037 But in particular, 00:02:02.677 --> 00:02:04.597 I would highlight outcome standard four 00:02:04.638 --> 00:02:06.697 point three in risk management. 00:02:06.998 --> 00:02:08.959 Any risks to students, 00:02:08.998 --> 00:02:12.219 to our personnel or to our organisation 00:02:12.699 --> 00:02:14.080 are organised and managed. 00:02:14.100 --> 00:02:16.200 And we know that there are so many 00:02:16.281 --> 00:02:19.241 risks that are presented in terms of AI 00:02:19.401 --> 00:02:21.242 use, whether it's students using AI, 00:02:21.703 --> 00:02:22.443 for example, 00:02:22.663 --> 00:02:24.704 in terms of assisting with assessments, 00:02:25.104 --> 00:02:27.064 whether it's our personnel using AI, 00:02:27.245 --> 00:02:30.145 are we doing BYO AI? 00:02:30.686 --> 00:02:32.026 bring your own along, 00:02:32.407 --> 00:02:35.248 or are they actually using tested and 00:02:35.288 --> 00:02:38.170 approved solutions that the organisation 00:02:38.570 --> 00:02:42.453 is paying for, monitoring and managing? 00:02:42.832 --> 00:02:43.954 And then what are the risks for the 00:02:44.133 --> 00:02:45.414 organisation overall? 00:02:45.474 --> 00:02:47.656 Privacy risks, governance risks, 00:02:48.137 --> 00:02:50.557 implementation risks, compliance risks, 00:02:50.638 --> 00:02:51.778 reputational risks. 00:02:52.299 --> 00:02:53.520 And so the list goes on. 00:02:53.939 --> 00:02:54.580 So really, 00:02:54.600 --> 00:02:55.961 when we're thinking about outcome standard 00:02:56.002 --> 00:02:57.703 four point three in the first instance, 00:02:57.742 --> 00:02:59.764 we must make sure that we've got a 00:02:59.804 --> 00:03:02.605 handle on our usage. 00:03:02.887 --> 00:03:04.768 And when we think about the guidance for 00:03:04.788 --> 00:03:06.209 the Australian from the Australian 00:03:06.229 --> 00:03:06.549 government, 00:03:06.588 --> 00:03:08.229 twenty twenty five has been a very big 00:03:08.270 --> 00:03:08.570 year. 00:03:08.991 --> 00:03:11.233 As we know, in twenty nineteen, 00:03:11.272 --> 00:03:13.334 we had some of the first guidance, 00:03:13.354 --> 00:03:15.616 Australia's ethics principles, 00:03:15.996 --> 00:03:17.917 and then that was followed up with the 00:03:18.137 --> 00:03:21.539 advent of the voluntary safety standard. 00:03:21.900 --> 00:03:23.901 Now just be thinking about November, 00:03:23.980 --> 00:03:26.502 was the release of CHAT-GPT. 00:03:26.801 --> 00:03:32.043 So if we're sitting here now in , 00:03:28.401 --> 00:03:32.723 we're three years on from that release and 00:03:33.163 --> 00:03:35.784 counting and really taking some time to 00:03:35.824 --> 00:03:38.145 get the formal guidance in place. 00:03:38.586 --> 00:03:41.326 In , 00:03:39.825 --> 00:03:42.187 the Office of the Australian Information 00:03:42.227 --> 00:03:45.268 Commissioner looked at the privacy 00:03:45.328 --> 00:03:47.149 implications for AI and released. 00:03:47.609 --> 00:03:49.670 privacy considerations when using 00:03:49.711 --> 00:03:51.872 commercially available ai products and 00:03:51.893 --> 00:03:55.695 that was updated again in for a current 00:03:55.735 --> 00:03:58.117 version and then our national artificial 00:03:58.157 --> 00:04:00.599 intelligence center has released critical 00:04:00.620 --> 00:04:03.641 documentation through we now have the 00:04:03.681 --> 00:04:06.144 guidance for ai adoption these are the 00:04:06.204 --> 00:04:06.844 essential 00:04:07.405 --> 00:04:09.407 implementation requirements for all 00:04:09.467 --> 00:04:11.710 organisations in the Australian region, 00:04:11.770 --> 00:04:12.931 including VET providers, 00:04:13.031 --> 00:04:14.673 on what we need to be doing to 00:04:14.752 --> 00:04:17.255 ensure that we are implementing effective 00:04:17.495 --> 00:04:20.139 AI practices in our organisations. 00:04:20.199 --> 00:04:22.382 We also now have the National AI Plan 00:04:22.822 --> 00:04:24.483 and that is a very strong focus in 00:04:24.523 --> 00:04:27.024 skills development that is useful to be 00:04:27.084 --> 00:04:28.526 considering from a vet provider 00:04:28.545 --> 00:04:31.288 perspective and we also most recently have 00:04:31.608 --> 00:04:34.350 new guidance for business on being clear 00:04:34.910 --> 00:04:38.192 about ai generated content and what we 00:04:38.211 --> 00:04:40.394 should be doing in terms of labeling 00:04:40.874 --> 00:04:43.276 or having other transparency mechanisms in 00:04:43.297 --> 00:04:46.279 place for when we are using AI generated 00:04:46.341 --> 00:04:46.800 content. 00:04:46.860 --> 00:04:48.343 And when we think about vet providers, 00:04:48.742 --> 00:04:50.225 if we think about learning resources, 00:04:50.264 --> 00:04:52.406 assessment resources, marketing resources, 00:04:52.728 --> 00:04:55.992 images, videos, the list goes on, 00:04:56.151 --> 00:04:59.194 we may well be using much AI generated 00:04:59.235 --> 00:05:01.218 content within our operations. 00:05:01.697 --> 00:05:02.939 So again, 00:05:03.059 --> 00:05:04.439 twenty twenty five has been a really 00:05:04.480 --> 00:05:06.081 critical year towards the end of twenty 00:05:06.100 --> 00:05:09.343 twenty five with all of this key guidance 00:05:09.384 --> 00:05:11.485 coming out about now the effective 00:05:11.586 --> 00:05:13.846 implementation of AI for Australian 00:05:13.906 --> 00:05:15.848 organisations, including their providers. 00:05:16.966 --> 00:05:17.706 And as we know, 00:05:18.387 --> 00:05:20.809 we are operating in the era of 00:05:20.889 --> 00:05:21.689 self-assurance. 00:05:21.709 --> 00:05:23.531 This is the way that we manage our 00:05:23.591 --> 00:05:24.430 operations, 00:05:24.951 --> 00:05:27.874 including our AI usage for a focus on 00:05:27.913 --> 00:05:28.473 quality, 00:05:29.014 --> 00:05:31.836 including compliance to not only the 00:05:31.877 --> 00:05:33.197 twenty twenty five standards, 00:05:33.237 --> 00:05:36.540 but to other government guidance and for 00:05:36.819 --> 00:05:38.401 ongoing continuous improvement. 00:05:38.480 --> 00:05:40.382 And we have to have systems in place 00:05:40.802 --> 00:05:42.603 to critically examine our performance. 00:05:42.644 --> 00:05:43.545 What is it we're doing? 00:05:44.144 --> 00:05:46.807 with AI across our operations and also the 00:05:46.887 --> 00:05:47.327 outcomes. 00:05:47.367 --> 00:05:50.569 What impact is our AI usage having in 00:05:50.750 --> 00:05:51.550 our operations? 00:05:52.071 --> 00:05:54.112 And we know with outcome standard four 00:05:54.132 --> 00:05:55.934 point four in the twenty twenty five 00:05:55.973 --> 00:05:56.415 standards, 00:05:56.454 --> 00:05:57.654 when we're thinking about continuous 00:05:57.716 --> 00:05:58.235 improvement, 00:05:58.636 --> 00:06:00.997 we're required to have in place systematic 00:06:01.218 --> 00:06:02.559 monitoring and evaluation. 00:06:02.598 --> 00:06:04.500 We need to understand what is the outcome. 00:06:05.000 --> 00:06:06.442 or the outcomes that we're aiming to 00:06:06.502 --> 00:06:07.802 achieve with AI usage? 00:06:08.163 --> 00:06:09.983 And what indicators are we going to put 00:06:10.004 --> 00:06:12.386 in place to measure those outcomes? 00:06:12.446 --> 00:06:14.747 What data and feedback are we going to 00:06:14.767 --> 00:06:15.427 be collecting? 00:06:15.807 --> 00:06:17.389 And how are we going to be monitoring 00:06:17.488 --> 00:06:20.170 the AI usage on an ongoing basis to 00:06:20.230 --> 00:06:23.012 ensure that we are achieving the outcomes 00:06:23.173 --> 00:06:24.053 that we're expecting? 00:06:24.613 --> 00:06:26.954 And then at periodic reviews, 00:06:27.014 --> 00:06:29.355 how are we going to evaluate the 00:06:29.396 --> 00:06:31.877 effectiveness of our AI usage? 00:06:32.317 --> 00:06:33.637 We need to be engaging with our 00:06:33.697 --> 00:06:35.619 stakeholders in those monitoring and 00:06:35.639 --> 00:06:37.680 evaluation activities to be getting their 00:06:37.839 --> 00:06:40.081 advice and feedback on what we're doing. 00:06:40.100 --> 00:06:41.920 And then we need to be communicating and 00:06:41.961 --> 00:06:44.622 reporting all of those findings back to 00:06:44.682 --> 00:06:45.702 our key stakeholders, 00:06:45.762 --> 00:06:47.244 including our governing persons, 00:06:47.564 --> 00:06:49.324 so that we can use that information for 00:06:49.384 --> 00:06:50.245 our decision making, 00:06:50.865 --> 00:06:53.165 and for improvements to our systems and 00:06:53.245 --> 00:06:54.387 our services delivery. 00:06:54.766 --> 00:06:55.226 So there's a very, 00:06:55.247 --> 00:06:57.387 very high bar there with systematic 00:06:57.447 --> 00:06:59.408 monitoring and evaluation across all of 00:06:59.428 --> 00:07:01.730 our operating functions within our VET 00:07:01.771 --> 00:07:04.632 provider and importantly, obviously, 00:07:04.791 --> 00:07:07.994 with our use of AI as well. 00:07:08.454 --> 00:07:08.553 Now, 00:07:08.574 --> 00:07:10.254 I'm going to pause there and we'll get 00:07:10.334 --> 00:07:11.755 into the AI pack. 00:07:12.935 --> 00:07:15.237 And there's some key documentation that I 00:07:15.276 --> 00:07:17.259 want to be looking at here as part 00:07:17.298 --> 00:07:17.798 of the pack. 00:07:17.838 --> 00:07:20.961 We're going to kick off with the area 00:07:21.341 --> 00:07:23.064 of governance. 00:07:23.504 --> 00:07:25.326 So let me just bring that pack back 00:07:25.386 --> 00:07:26.067 up here again. 00:07:27.187 --> 00:07:28.247 And we're going to be looking at the 00:07:28.327 --> 00:07:29.749 area of governance and planning. 00:07:29.889 --> 00:07:31.069 Then we're going to be considering 00:07:31.288 --> 00:07:33.348 templates and tools for implementation. 00:07:33.928 --> 00:07:35.050 And then we're going to be considering 00:07:35.110 --> 00:07:36.610 templates and tools as well for that 00:07:36.730 --> 00:07:38.610 monitoring and evaluation function. 00:07:38.629 --> 00:07:40.831 All of these are included within our AI 00:07:41.071 --> 00:07:41.810 adoption pack. 00:07:42.091 --> 00:07:42.850 Before I do that, 00:07:42.990 --> 00:07:44.351 I do just want to mention as well, 00:07:44.391 --> 00:07:46.351 we've also included a folder with all of 00:07:46.391 --> 00:07:47.512 the supporting documents. 00:07:47.971 --> 00:07:48.913 So if you jump in there, 00:07:49.132 --> 00:07:52.632 all of that government guidance that I 00:07:52.673 --> 00:07:55.153 mentioned during the presentation and more 00:07:55.173 --> 00:07:55.574 tools 00:07:56.033 --> 00:07:56.754 are included, 00:07:56.973 --> 00:07:58.774 government tools are included in that 00:07:58.795 --> 00:08:00.716 supporting documentation folder as well. 00:08:01.336 --> 00:08:02.815 So let's click off with governance and 00:08:02.836 --> 00:08:03.257 planning. 00:08:03.576 --> 00:08:05.297 And there's probably no surprises here. 00:08:05.497 --> 00:08:07.338 We're starting off with a stakeholder 00:08:07.497 --> 00:08:08.538 impact analysis. 00:08:08.937 --> 00:08:12.899 So how do our stakeholders feel about the 00:08:12.939 --> 00:08:16.680 implementation of AI and what impacts may 00:08:16.920 --> 00:08:18.701 this implementation have on our 00:08:18.742 --> 00:08:19.322 stakeholders? 00:08:19.701 --> 00:08:20.622 Remembering quality, 00:08:20.903 --> 00:08:22.264 quality services delivery, 00:08:22.303 --> 00:08:25.326 quality is fulfilling stated and implied 00:08:25.365 --> 00:08:25.766 needs, 00:08:25.826 --> 00:08:28.148 is fulfilling the requirements that we've 00:08:28.208 --> 00:08:30.108 identified from our stakeholders, 00:08:30.129 --> 00:08:31.750 the very definition under the 00:08:31.810 --> 00:08:32.910 international standard. 00:08:33.511 --> 00:08:35.753 So we've got a stakeholder analysis tool 00:08:35.793 --> 00:08:37.354 looking at who our stakeholders are, 00:08:37.394 --> 00:08:38.695 their types, their interests, 00:08:38.735 --> 00:08:40.657 their influence, their expectations. 00:08:41.236 --> 00:08:42.918 What are the goals and communication 00:08:42.957 --> 00:08:43.278 needs? 00:08:43.317 --> 00:08:45.259 How might they influence our strategy? 00:08:45.820 --> 00:08:47.721 and then we're taking an impact 00:08:47.782 --> 00:08:48.302 assessment. 00:08:48.341 --> 00:08:51.683 We're using the AI screening tool or we're 00:08:51.764 --> 00:08:54.066 using our standard risk assessment to do 00:08:54.086 --> 00:08:56.888 an impact assessment on how will this use 00:08:56.927 --> 00:08:59.850 of AI impact our stakeholders and then be 00:08:59.889 --> 00:09:01.211 thinking about the effective 00:09:01.270 --> 00:09:04.253 implementation of AI from that basis. 00:09:05.234 --> 00:09:05.514 So again, 00:09:05.533 --> 00:09:06.913 don't want to spend too much more time 00:09:06.933 --> 00:09:07.274 on that, 00:09:07.754 --> 00:09:10.014 but really starting off with that 00:09:10.054 --> 00:09:11.676 stakeholder expectation. 00:09:12.056 --> 00:09:14.475 We've also included a stakeholder survey 00:09:14.596 --> 00:09:15.336 on the use of AI. 00:09:15.976 --> 00:09:16.096 Now, 00:09:16.136 --> 00:09:17.856 this could be used as part of your 00:09:17.937 --> 00:09:18.797 implementation. 00:09:19.197 --> 00:09:22.038 This obviously is to be used as part 00:09:22.057 --> 00:09:23.557 of your monitoring and evaluation 00:09:23.597 --> 00:09:24.058 activities, 00:09:24.379 --> 00:09:25.958 but we're including it here in governance 00:09:26.019 --> 00:09:26.918 upfront, again, 00:09:27.339 --> 00:09:29.600 to get that information from stakeholders 00:09:29.620 --> 00:09:30.559 in the first instance, 00:09:30.580 --> 00:09:32.660 to support your governance and planning 00:09:32.701 --> 00:09:33.921 for the implementation. 00:09:34.520 --> 00:09:36.902 And again, it's a stakeholder survey, 00:09:36.942 --> 00:09:38.043 so it could be getting used with 00:09:38.102 --> 00:09:38.643 personnel, 00:09:38.682 --> 00:09:39.884 could be getting used with students, 00:09:39.903 --> 00:09:41.203 with employers, with industry, 00:09:41.244 --> 00:09:43.225 with community representatives, et cetera. 00:09:43.706 --> 00:09:45.245 And it's just a couple of pages there 00:09:45.546 --> 00:09:48.288 thinking about key aspects of the 00:09:48.347 --> 00:09:52.070 stakeholders' use and expectations around 00:09:52.090 --> 00:09:52.649 AI. 00:09:53.070 --> 00:09:55.071 We've got the standard four-point Likert 00:09:55.091 --> 00:09:57.951 scale there to make that quantitative 00:09:57.991 --> 00:10:00.253 analysis really easy and to be gathering 00:10:00.293 --> 00:10:01.974 the data that you need to be supporting 00:10:01.994 --> 00:10:03.274 that stakeholder analysis. 00:10:03.794 --> 00:10:06.316 as i mentioned that survey is obviously 00:10:06.336 --> 00:10:08.356 then also available for use with both 00:10:08.437 --> 00:10:10.258 implementation and also monitoring and 00:10:10.278 --> 00:10:12.658 evaluation once we've got all of that 00:10:12.698 --> 00:10:14.759 information in place we can be putting 00:10:14.818 --> 00:10:17.480 together our ai policy and it is an 00:10:17.539 --> 00:10:20.461 expectation that all organizations will 00:10:20.522 --> 00:10:22.682 have an ai policy in place part of 00:10:23.182 --> 00:10:25.082 our guidance for ai adoption from the 00:10:25.123 --> 00:10:27.464 national ai center and i won't spend too 00:10:27.484 --> 00:10:29.404 much time on this obviously we've got a 00:10:29.424 --> 00:10:31.446 policy format so we've got the purpose 00:10:31.985 --> 00:10:33.326 the scope of the policy, 00:10:33.386 --> 00:10:34.488 who it applies to. 00:10:34.888 --> 00:10:36.448 With all of our VETA policies, 00:10:36.509 --> 00:10:38.389 and we recommend for all VET providers for 00:10:38.409 --> 00:10:39.431 your policy positions, 00:10:39.490 --> 00:10:40.552 include a rationale. 00:10:40.711 --> 00:10:43.173 Make sure that you've got that summary or 00:10:43.193 --> 00:10:44.274 that justification, 00:10:44.293 --> 00:10:46.416 that rationale for how you're approaching 00:10:46.875 --> 00:10:48.657 that particular operating function of your 00:10:48.697 --> 00:10:49.357 organisation. 00:10:49.378 --> 00:10:49.557 Very, 00:10:49.577 --> 00:10:52.240 very handy to give that summary for all 00:10:52.539 --> 00:10:54.120 of your policy users, 00:10:54.461 --> 00:10:57.123 including both internal but also external 00:10:57.363 --> 00:10:59.065 and right through to your regulator 00:10:59.164 --> 00:10:59.565 auditors. 00:10:59.625 --> 00:11:00.926 So get that rationale in there. 00:11:02.403 --> 00:11:05.745 We've got the governance of AI and key 00:11:05.784 --> 00:11:07.504 components here for governance, again, 00:11:07.625 --> 00:11:09.946 aligned directly to the Australian 00:11:09.985 --> 00:11:11.746 government guidance for the 00:11:11.767 --> 00:11:12.587 implementation. 00:11:12.988 --> 00:11:16.129 We're mentioning here the use of an AI 00:11:16.168 --> 00:11:17.429 register to be tracking. 00:11:18.190 --> 00:11:20.431 and keeping that monitoring going of all 00:11:20.610 --> 00:11:22.650 AI that's been approved and has been used 00:11:22.730 --> 00:11:23.652 in the organisation. 00:11:24.091 --> 00:11:25.792 We've got the roles and responsibilities 00:11:25.831 --> 00:11:26.731 for AI use, 00:11:27.113 --> 00:11:29.732 how you go about implementing new AI 00:11:29.753 --> 00:11:30.153 tools, 00:11:30.212 --> 00:11:32.094 including the use of the AI screening 00:11:32.134 --> 00:11:32.453 tool, 00:11:32.953 --> 00:11:34.855 how you go about the management of AI 00:11:34.894 --> 00:11:38.096 generated content directly aligned to the 00:11:38.155 --> 00:11:39.916 new guidance from the Australian 00:11:39.956 --> 00:11:42.397 government for the transparency measures 00:11:42.417 --> 00:11:43.777 that need to be in place for your 00:11:43.937 --> 00:11:45.738 use of AI generated content. 00:11:46.717 --> 00:11:48.379 You can see information there on the 00:11:48.399 --> 00:11:49.519 transparency measures, 00:11:49.720 --> 00:11:51.561 also incident reporting when something 00:11:51.581 --> 00:11:54.202 goes wrong and that standard monitoring 00:11:54.222 --> 00:11:55.884 and evaluation approach with your policy 00:11:55.923 --> 00:11:56.323 positions. 00:11:56.364 --> 00:11:57.725 You can see there we refer out, 00:11:58.085 --> 00:11:59.625 we're not repeating huge amounts of 00:11:59.666 --> 00:12:01.506 monitoring and evaluation information that 00:12:01.547 --> 00:12:02.548 should be in your policy, 00:12:02.947 --> 00:12:05.168 your quality policy and procedures. 00:12:05.509 --> 00:12:07.971 So just reference all the other policies 00:12:07.990 --> 00:12:09.591 that you have across there and make sure 00:12:09.711 --> 00:12:10.613 that that's covered off. 00:12:11.033 --> 00:12:12.594 So you do need to have your AI 00:12:12.614 --> 00:12:13.754 policy position in place. 00:12:13.815 --> 00:12:15.196 Hopefully you found that useful. 00:12:15.716 --> 00:12:19.197 We're also including a risk matrix on AI 00:12:19.217 --> 00:12:19.678 adoption. 00:12:20.158 --> 00:12:21.940 And this is a collection of all of 00:12:21.960 --> 00:12:24.662 the government guidance on the common 00:12:24.741 --> 00:12:26.822 risks for use of AI. 00:12:27.023 --> 00:12:28.524 And it covers here, as you can see, 00:12:28.865 --> 00:12:31.466 the risk, the effect, the risk rating, 00:12:31.746 --> 00:12:33.607 and then the mitigating actions that you 00:12:33.648 --> 00:12:34.347 might consider. 00:12:34.849 --> 00:12:36.269 And we've got the adoption of AI. 00:12:37.230 --> 00:12:40.631 we have both for the organisation, 00:12:40.892 --> 00:12:43.073 we've got the system purpose deployment of 00:12:43.173 --> 00:12:45.554 AI, we're talking about data management, 00:12:45.595 --> 00:12:47.296 system design, security, 00:12:47.917 --> 00:12:49.918 communications with our customers and 00:12:49.957 --> 00:12:50.678 stakeholders, 00:12:51.558 --> 00:12:53.900 using AI enabled products and so the list 00:12:53.941 --> 00:12:54.360 goes on. 00:12:54.860 --> 00:12:55.221 So again, 00:12:55.240 --> 00:12:57.802 we've collated all of the common risks 00:12:57.863 --> 00:13:01.065 that have been identified in various 00:13:01.085 --> 00:13:02.145 government documentation. 00:13:02.605 --> 00:13:03.606 you might find that really, 00:13:03.647 --> 00:13:06.230 really useful if you're populating your 00:13:06.250 --> 00:13:08.092 risk management plan or indeed your risk 00:13:08.153 --> 00:13:10.676 register in capturing this area of AI. 00:13:12.418 --> 00:13:13.899 The other governance and planning tool 00:13:13.918 --> 00:13:16.620 that we've included is our generative AI 00:13:16.721 --> 00:13:18.081 implementation strategy. 00:13:18.501 --> 00:13:20.523 And this is a strategy that's sitting 00:13:20.562 --> 00:13:21.864 across the organisation. 00:13:21.903 --> 00:13:23.705 I'm not going to spend too much time 00:13:23.745 --> 00:13:25.525 on this, but just in general, 00:13:25.905 --> 00:13:27.866 the format for all of our planning 00:13:27.907 --> 00:13:29.768 documentation strategies and plans, 00:13:29.807 --> 00:13:31.269 we should have, again, the purpose, 00:13:31.288 --> 00:13:32.870 the scope, the vision, 00:13:33.210 --> 00:13:35.251 the objectives or the goals that we're 00:13:35.272 --> 00:13:36.091 aiming to achieve. 00:13:36.451 --> 00:13:38.754 We also, if we're thinking about risk, 00:13:38.813 --> 00:13:40.094 we need to have our risk management 00:13:40.134 --> 00:13:40.754 section in there. 00:13:41.315 --> 00:13:43.115 outcome standard four point three outcome 00:13:43.134 --> 00:13:44.635 standard four point four systematic 00:13:44.676 --> 00:13:46.135 monitoring and evaluation how are you 00:13:46.176 --> 00:13:48.436 going to be monitoring and evaluating this 00:13:48.535 --> 00:13:51.136 policy and in the middle you should have 00:13:51.197 --> 00:13:53.998 the the key components of what the topic 00:13:54.097 --> 00:13:55.538 is if it's the risk management plan it's 00:13:55.557 --> 00:13:57.177 going to be talking about risk in this 00:13:57.217 --> 00:14:00.078 case it's the generative ai plan strategy 00:14:00.119 --> 00:14:01.839 so we're going to be talking about our 00:14:01.899 --> 00:14:04.740 generative ai strategic initiatives so if 00:14:04.759 --> 00:14:05.940 i just quickly jump there 00:14:07.351 --> 00:14:09.393 we can see just that format there, 00:14:09.592 --> 00:14:10.673 what we are doing, 00:14:10.732 --> 00:14:12.514 what are the strategic initiatives that 00:14:12.533 --> 00:14:14.234 we're implementing in the organisation and 00:14:14.274 --> 00:14:16.615 what are we aiming to achieve with those 00:14:16.654 --> 00:14:18.235 initiatives and what performance and 00:14:18.316 --> 00:14:20.735 outcome indicators, self-assurance, 00:14:20.796 --> 00:14:21.996 critical examination, 00:14:23.037 --> 00:14:24.998 of our performance and our outcomes, 00:14:25.378 --> 00:14:26.958 outcome standard four point four, 00:14:27.099 --> 00:14:29.160 our systematic monitoring and evaluation. 00:14:29.480 --> 00:14:30.941 We've got our performance and outcome 00:14:30.961 --> 00:14:33.283 indicators specified on how we're going to 00:14:33.322 --> 00:14:37.565 measure whether our AI use is achieving a 00:14:37.644 --> 00:14:38.946 quality implementation. 00:14:39.485 --> 00:14:39.725 So again, 00:14:39.745 --> 00:14:41.148 I won't spend too much more time on 00:14:41.168 --> 00:14:41.388 that, 00:14:41.688 --> 00:14:42.950 but hopefully you can see they're very, 00:14:42.970 --> 00:14:44.191 very robust approach, 00:14:44.490 --> 00:14:45.932 talking with our stakeholders, 00:14:46.413 --> 00:14:47.715 identifying the risks, 00:14:48.316 --> 00:14:50.057 implementing a policy position, 00:14:50.238 --> 00:14:52.519 and then planning out with our generative 00:14:52.600 --> 00:14:53.380 AI strategy, 00:14:53.441 --> 00:14:55.043 how we're going to go about the effective 00:14:55.102 --> 00:14:57.926 implementation within our organisation. 00:14:59.453 --> 00:14:59.995 From there, 00:15:00.755 --> 00:15:03.278 if I then jump into that implementation, 00:15:03.318 --> 00:15:05.620 there's a whole range of tools here that 00:15:05.961 --> 00:15:08.104 I want to just quickly mention that are 00:15:08.124 --> 00:15:09.645 included in the pack. 00:15:09.806 --> 00:15:10.746 First one here, 00:15:10.787 --> 00:15:12.929 we've got a risk assessment record. 00:15:12.970 --> 00:15:14.792 This is our standard risk assessment 00:15:14.812 --> 00:15:15.231 approach. 00:15:15.272 --> 00:15:18.275 It is aligned to ISO-XXX for risk 00:15:18.316 --> 00:15:19.116 management as well. 00:15:19.636 --> 00:15:22.078 and is helping us to do any kind 00:15:22.119 --> 00:15:23.900 of risk assessment in the organisation. 00:15:23.941 --> 00:15:24.880 Obviously, in this case, 00:15:25.302 --> 00:15:27.363 we'd be looking at the risk assessment on 00:15:27.383 --> 00:15:27.984 the use of AI. 00:15:28.004 --> 00:15:31.066 So we're covering off the general use 00:15:31.105 --> 00:15:31.447 case, 00:15:31.547 --> 00:15:33.528 we're then looking at the likelihood, 00:15:33.548 --> 00:15:34.349 the consequence, 00:15:34.428 --> 00:15:37.071 we're coming up with a risk rating for 00:15:37.091 --> 00:15:38.471 the implementation of the tool, 00:15:38.873 --> 00:15:40.433 and then we're considering the treatment 00:15:40.474 --> 00:15:42.916 or the mitigating actions to ensure that 00:15:42.956 --> 00:15:44.317 we can implement that 00:15:44.917 --> 00:15:45.859 tool effectively, 00:15:46.339 --> 00:15:48.260 looking at the hierarchy of control there 00:15:48.662 --> 00:15:50.123 and then the treatment plan for that 00:15:50.163 --> 00:15:53.447 particular risk to get to the residual 00:15:53.486 --> 00:15:55.408 risk rating and then the monitoring and 00:15:55.448 --> 00:15:56.850 review arrangements of that risk. 00:15:56.890 --> 00:15:59.092 So a standard risk assessment process that 00:15:59.133 --> 00:16:02.957 we'd be applying for each AI tool before 00:16:03.018 --> 00:16:05.480 it's implemented within the organisation. 00:16:06.061 --> 00:16:07.802 The other big assessment tool that I just 00:16:07.841 --> 00:16:09.783 quickly mentioned here as well is the 00:16:09.822 --> 00:16:11.562 privacy impact assessment tool. 00:16:11.844 --> 00:16:13.844 So it is an expectation for major 00:16:13.884 --> 00:16:15.304 implementations that we would be 00:16:15.365 --> 00:16:17.365 undertaking a privacy impact assessment as 00:16:17.405 --> 00:16:17.645 well. 00:16:18.066 --> 00:16:20.346 This one is aligned to the expectations 00:16:20.386 --> 00:16:22.128 from the Office of the Australian 00:16:22.168 --> 00:16:23.168 Information Commissioner, 00:16:23.208 --> 00:16:23.908 as you would expect. 00:16:24.548 --> 00:16:26.971 And again, we've got a key detail there, 00:16:27.030 --> 00:16:30.133 not only on what the key questions are 00:16:30.153 --> 00:16:31.072 that we should be asking, 00:16:31.452 --> 00:16:33.815 but we've got key prompt information here 00:16:33.835 --> 00:16:35.515 as you're completing your privacy impact 00:16:35.556 --> 00:16:37.537 assessment on how you're going about the 00:16:37.616 --> 00:16:38.418 use of that tool. 00:16:38.918 --> 00:16:39.378 So again, 00:16:39.398 --> 00:16:41.019 won't spend too much more time on that, 00:16:41.078 --> 00:16:42.860 but just to note that the tool is 00:16:42.899 --> 00:16:43.179 there. 00:16:43.640 --> 00:16:46.903 So just to summarise that expectation that 00:16:47.062 --> 00:16:49.683 if you are implementing a new AI tool 00:16:49.703 --> 00:16:50.884 within your organisation, 00:16:50.965 --> 00:16:52.666 that you're undertaking a stakeholder 00:16:52.706 --> 00:16:53.586 impact analysis, 00:16:54.267 --> 00:16:56.969 that you're undertaking a privacy impact 00:16:57.269 --> 00:16:59.630 analysis and that you're undertaking your 00:16:59.750 --> 00:17:00.671 risk analysis. 00:17:00.711 --> 00:17:03.153 So those three tools going together are 00:17:03.192 --> 00:17:05.074 really bringing down, 00:17:05.114 --> 00:17:07.035 working through and bringing down to a 00:17:07.095 --> 00:17:08.997 final decision around whether you can 00:17:09.037 --> 00:17:12.239 effectively implement safely this AI for 00:17:12.259 --> 00:17:13.759 the use case that you're talking about. 00:17:15.059 --> 00:17:15.519 From there, 00:17:15.701 --> 00:17:18.061 we've got some of the other tools that 00:17:18.102 --> 00:17:18.862 I mentioned before. 00:17:19.241 --> 00:17:21.423 We have an AI implementation checklist. 00:17:21.482 --> 00:17:23.943 This is based on the guidance for AI 00:17:23.963 --> 00:17:26.164 adoption from the National AI Centre. 00:17:26.566 --> 00:17:27.546 So as you can see there, 00:17:27.945 --> 00:17:29.606 and it is an extensive checklist. 00:17:29.666 --> 00:17:32.969 This one is covering off six pages of 00:17:33.048 --> 00:17:35.329 detailed requirements around those six 00:17:35.490 --> 00:17:37.891 essential functions that must be completed 00:17:37.951 --> 00:17:40.051 for your AI implementation, 00:17:40.491 --> 00:17:41.752 deciding who is accountable, 00:17:42.292 --> 00:17:43.313 and the requirements here, 00:17:43.333 --> 00:17:44.492 and you can see the format. 00:17:44.874 --> 00:17:47.193 It is designed as a checklist for your 00:17:47.233 --> 00:17:48.114 implementation. 00:17:48.413 --> 00:17:49.114 It's also very, 00:17:49.134 --> 00:17:51.494 very handy as a checklist to monitor or 00:17:51.535 --> 00:17:54.454 to audit what's happened if you've perhaps 00:17:54.515 --> 00:17:57.016 done AI implementations without following 00:17:57.215 --> 00:17:59.316 all of these steps prior to the guidance 00:17:59.336 --> 00:18:00.076 being available. 00:18:00.635 --> 00:18:01.436 Who is accountable? 00:18:01.797 --> 00:18:03.376 And we've got the requirements there for 00:18:03.396 --> 00:18:04.136 you to consider. 00:18:04.817 --> 00:18:06.897 Understanding the impacts and planning out 00:18:06.978 --> 00:18:07.958 that implementation. 00:18:08.897 --> 00:18:10.920 measuring and managing the risks of the 00:18:10.940 --> 00:18:11.759 implementation, 00:18:12.681 --> 00:18:14.422 sharing the essential information with 00:18:14.442 --> 00:18:15.323 your stakeholders, 00:18:15.383 --> 00:18:17.663 including our students primarily, 00:18:18.625 --> 00:18:23.208 testing and monitoring the implementation 00:18:23.228 --> 00:18:24.209 on an ongoing basis, 00:18:25.009 --> 00:18:26.450 making sure that it's still working the 00:18:26.470 --> 00:18:28.151 way that we're expecting it to be working 00:18:28.290 --> 00:18:30.771 and finally maintaining that human control 00:18:30.912 --> 00:18:34.054 so there is that ongoing human oversight 00:18:34.094 --> 00:18:36.615 of what is going on with the ai 00:18:36.655 --> 00:18:39.277 implementation so very very useful tool 00:18:39.616 --> 00:18:43.700 that is used for every implementation when 00:18:43.720 --> 00:18:45.861 you get to that implementation stage of 00:18:45.901 --> 00:18:48.102 your ai usage we've also got the ai 00:18:48.122 --> 00:18:50.584 register there so what we're talking about 00:18:52.006 --> 00:18:54.467 is actually tracking and monitoring all of 00:18:54.527 --> 00:18:57.326 the AI tools in use in the organisation. 00:18:57.988 --> 00:18:58.847 Let me just bring that up. 00:18:59.567 --> 00:19:00.327 And you can see here, 00:19:00.367 --> 00:19:02.489 just to fly through this quickly, the ID, 00:19:02.729 --> 00:19:03.288 the date, 00:19:03.729 --> 00:19:06.170 the name and version of the AI being 00:19:06.349 --> 00:19:06.589 used, 00:19:06.710 --> 00:19:08.470 who is the owner in the organisation, 00:19:08.509 --> 00:19:09.510 what's the status, 00:19:09.570 --> 00:19:10.871 we're including dropdowns, 00:19:11.510 --> 00:19:15.276 relevant to the uh to the register 00:19:15.296 --> 00:19:17.759 wherever possible what's the source of the 00:19:17.858 --> 00:19:19.922 ai what goals are we aiming for the 00:19:19.961 --> 00:19:22.164 intended use cases and the links to our 00:19:22.204 --> 00:19:26.329 strategic objectives so again that link 00:19:26.369 --> 00:19:27.791 back to our strategy 00:19:28.613 --> 00:19:31.835 the known limitations of the solution and 00:19:31.875 --> 00:19:32.835 prohibited use, 00:19:32.914 --> 00:19:35.356 what can't we do with the solution, 00:19:35.517 --> 00:19:37.698 any foreseeable misuse that we need to be 00:19:37.738 --> 00:19:38.218 managing, 00:19:38.758 --> 00:19:40.638 the data sources and types we're going to 00:19:40.659 --> 00:19:41.140 be using, 00:19:41.180 --> 00:19:43.800 the risk rating that we've come up with 00:19:43.840 --> 00:19:45.882 and who's the risk owner that's going to 00:19:45.902 --> 00:19:47.762 be monitoring the risk of the solution, 00:19:48.182 --> 00:19:50.284 who are the stakeholders that have been 00:19:50.304 --> 00:19:52.184 affected and the screening that we've 00:19:52.226 --> 00:19:53.945 undertaken and the controls that we've 00:19:54.006 --> 00:19:55.666 implemented to make sure that this is 00:19:55.686 --> 00:19:56.548 going to be safe. 00:19:57.127 --> 00:19:58.988 and is going to deliver an effective 00:19:59.048 --> 00:20:00.969 outcome as we expect it to do. 00:20:01.528 --> 00:20:02.368 And then finally, 00:20:02.548 --> 00:20:05.190 the residual risk rating and the review 00:20:05.230 --> 00:20:06.869 dates for that ongoing review. 00:20:07.410 --> 00:20:08.611 So again, as you would expect, 00:20:08.750 --> 00:20:10.391 fairly straightforward register, 00:20:10.691 --> 00:20:12.172 keeping a track of what's going on, 00:20:12.231 --> 00:20:13.872 allowing us to monitor and regularly 00:20:13.951 --> 00:20:17.252 review what's happening with our AI 00:20:17.393 --> 00:20:18.272 implementations. 00:20:23.000 --> 00:20:26.143 The next one I want to quickly mention 00:20:26.202 --> 00:20:28.144 here is the AI screening tool. 00:20:28.223 --> 00:20:30.605 This is a tool from the National AI 00:20:30.645 --> 00:20:31.006 Centre. 00:20:31.266 --> 00:20:33.027 You can see it was released in October, 00:20:33.067 --> 00:20:33.968 twenty twenty five. 00:20:34.407 --> 00:20:35.209 And this is very, 00:20:35.249 --> 00:20:37.750 very straightforward tool to be 00:20:37.849 --> 00:20:39.811 considering, again, the risk level. 00:20:40.731 --> 00:20:43.273 and expectations before you implement any 00:20:43.473 --> 00:20:44.355 AI solution. 00:20:44.375 --> 00:20:46.135 I'll just quickly run through the tool. 00:20:46.656 --> 00:20:48.538 It's based on some key questions here. 00:20:49.058 --> 00:20:50.960 Does the AI system access, 00:20:51.000 --> 00:20:52.661 process or handle any personal, 00:20:52.701 --> 00:20:54.462 sensitive or confidential information? 00:20:56.262 --> 00:20:58.304 Does the AI interact directly with end 00:20:58.463 --> 00:20:58.844 users? 00:20:59.564 --> 00:21:01.604 Does it act autonomously at scale? 00:21:02.545 --> 00:21:04.605 Is it making or supporting decisions that 00:21:04.625 --> 00:21:06.086 are affecting your stakeholders? 00:21:06.746 --> 00:21:09.346 Does the AI system operate in a regulated 00:21:09.465 --> 00:21:11.747 area or have legal effect on individuals? 00:21:12.186 --> 00:21:14.367 Would harm from a wrong decision be 00:21:14.407 --> 00:21:15.667 difficult to contest, 00:21:15.728 --> 00:21:17.647 reverse or provide redress for? 00:21:18.229 --> 00:21:20.128 And is it designed for multiple purposes 00:21:20.429 --> 00:21:23.289 or easily adapted beyond its intended use? 00:21:24.230 --> 00:21:26.332 If you answer no to all questions, 00:21:26.791 --> 00:21:30.714 then you can continue for implementation 00:21:31.276 --> 00:21:33.698 within your standard organisational 00:21:33.738 --> 00:21:34.238 approach. 00:21:34.699 --> 00:21:36.380 If you answer yes to any of those 00:21:36.420 --> 00:21:36.940 questions, 00:21:36.960 --> 00:21:39.281 then it's hard to think about an AI 00:21:39.321 --> 00:21:41.304 implementation that wouldn't have yes to 00:21:41.423 --> 00:21:42.345 one or more of those. 00:21:42.825 --> 00:21:44.586 then you need to do that full 00:21:45.026 --> 00:21:46.846 implementation checklist that I just 00:21:46.866 --> 00:21:49.387 demonstrated that we've pulled together 00:21:49.468 --> 00:21:51.949 from those six essential requirements for 00:21:52.328 --> 00:21:54.269 the guidance for AI adoption. 00:21:55.461 --> 00:21:57.042 So we've got a screening tool there that 00:21:57.083 --> 00:21:58.384 gives you the starting point. 00:21:59.005 --> 00:22:00.246 And then we've got that extensive 00:22:00.286 --> 00:22:01.968 implementation checklist that we're 00:22:01.988 --> 00:22:03.288 suggesting you're probably going to be 00:22:03.388 --> 00:22:05.590 using with every single AI implementation, 00:22:06.011 --> 00:22:07.212 because you're going to have one or more 00:22:07.252 --> 00:22:10.056 yeses in that screening tool where you've 00:22:10.455 --> 00:22:13.500 to help guide and manage that AI 00:22:13.559 --> 00:22:15.000 implementation effectively in the 00:22:15.060 --> 00:22:15.741 organisation. 00:22:17.170 --> 00:22:17.349 Now, 00:22:17.490 --> 00:22:22.934 we do have a range of other implementation 00:22:23.115 --> 00:22:25.096 examples included in the pack, 00:22:25.136 --> 00:22:26.038 and I think this is very, 00:22:26.077 --> 00:22:28.580 very useful for VET providers. 00:22:28.740 --> 00:22:29.521 For example, 00:22:30.461 --> 00:22:32.904 we've got the example AI inclusions for 00:22:32.924 --> 00:22:33.865 your privacy policy. 00:22:34.566 --> 00:22:38.048 So we've got the key privacy information 00:22:38.068 --> 00:22:38.308 there. 00:22:38.328 --> 00:22:40.750 We've actually included our full example 00:22:40.770 --> 00:22:42.633 privacy policy, eleven pages. 00:22:43.133 --> 00:22:45.075 And within that privacy policy, 00:22:45.115 --> 00:22:47.997 you'll see key sections that are talking 00:22:48.017 --> 00:22:50.719 about AI usage in the relevant Australian 00:22:50.759 --> 00:22:52.080 Privacy Principle areas. 00:22:52.361 --> 00:22:54.001 If I just jump right to the bottom 00:22:54.061 --> 00:22:54.321 here, 00:22:54.342 --> 00:22:55.784 we should be able to find a section 00:22:56.203 --> 00:22:58.586 that jumps out on that. 00:23:01.027 --> 00:23:04.128 So there I've got AI governance and 00:23:04.169 --> 00:23:04.709 assurance. 00:23:04.729 --> 00:23:08.530 There's a whole section for insertion in 00:23:08.550 --> 00:23:09.451 the privacy policy. 00:23:09.471 --> 00:23:10.412 But again, as I mentioned, 00:23:10.832 --> 00:23:12.452 the information is also sprinkled 00:23:12.472 --> 00:23:15.595 throughout the policy on how AI is 00:23:15.734 --> 00:23:18.096 impacting privacy arrangements within your 00:23:18.155 --> 00:23:18.836 organisation. 00:23:19.277 --> 00:23:21.778 So if you haven't yet addressed AI within 00:23:21.798 --> 00:23:22.758 your privacy policy, 00:23:22.798 --> 00:23:25.539 do make sure that you are undertaking 00:23:25.559 --> 00:23:25.759 that. 00:23:26.259 --> 00:23:28.161 We've also included an example, 00:23:28.340 --> 00:23:29.221 APP notice, 00:23:29.682 --> 00:23:31.222 This is where whenever you're collecting 00:23:31.262 --> 00:23:32.223 personal information, 00:23:32.284 --> 00:23:34.045 think about application for enrolment as 00:23:34.065 --> 00:23:34.664 an example, 00:23:35.105 --> 00:23:37.467 we are required to provide an APP notice 00:23:37.787 --> 00:23:39.548 and we have included in there a section 00:23:40.048 --> 00:23:43.830 on the use of AI that would be 00:23:43.891 --> 00:23:44.392 completed. 00:23:45.071 --> 00:23:47.433 we have also included example ai 00:23:47.473 --> 00:23:49.515 inclusions for your application for 00:23:49.535 --> 00:23:51.696 enrollment form so we've taken that a step 00:23:51.737 --> 00:23:54.077 further you are required to have your 00:23:54.137 --> 00:23:56.299 privacy notice in place as part of your 00:23:56.319 --> 00:23:58.421 application for enrollment and we've got 00:23:58.461 --> 00:24:00.362 some example information that you might 00:24:00.422 --> 00:24:03.884 take tailor and expand relevant to the ai 00:24:04.025 --> 00:24:06.066 use for your organization so that you make 00:24:06.105 --> 00:24:07.666 sure that you've got that covered at 00:24:07.767 --> 00:24:10.588 application stage that your students or at 00:24:10.608 --> 00:24:12.470 this point your applicants are being 00:24:12.509 --> 00:24:14.592 effectively informed about your ai use 00:24:15.972 --> 00:24:18.253 We've also got example inclusions for your 00:24:18.273 --> 00:24:19.114 assessment policy, 00:24:19.433 --> 00:24:22.135 thinking about academic integrity and the 00:24:22.256 --> 00:24:23.175 use of AI, 00:24:23.797 --> 00:24:26.698 permitted uses and prohibited uses overall 00:24:27.179 --> 00:24:30.180 across your organisation when it comes to 00:24:30.259 --> 00:24:32.141 student assessment and the students' 00:24:32.181 --> 00:24:33.021 responsibilities. 00:24:33.582 --> 00:24:34.163 So again, 00:24:34.182 --> 00:24:36.124 I won't spend too much time going through 00:24:36.163 --> 00:24:36.384 that, 00:24:36.463 --> 00:24:38.885 but if you haven't yet built AI into 00:24:38.925 --> 00:24:40.007 your assessment policy, 00:24:40.446 --> 00:24:42.929 then you might find that really useful. 00:24:43.409 --> 00:24:44.109 And similarly, 00:24:44.170 --> 00:24:46.050 we've got example inclusions for your 00:24:46.131 --> 00:24:48.692 assessment tools and the assessment tasks 00:24:48.732 --> 00:24:48.992 as well. 00:24:49.153 --> 00:24:50.714 So what instructions are you giving your 00:24:50.734 --> 00:24:53.497 students on what they can or can't do 00:24:53.537 --> 00:24:56.598 with AI as part of a specific assessment, 00:24:57.058 --> 00:24:58.921 including their declarations on what they 00:24:58.961 --> 00:25:00.021 have or haven't done with AI, 00:25:00.682 --> 00:25:02.442 and also what instructions are you giving 00:25:02.462 --> 00:25:03.183 your assessors 00:25:03.624 --> 00:25:05.045 as part of your assessment tools. 00:25:05.365 --> 00:25:07.846 That can also be adapted for instructions 00:25:07.865 --> 00:25:09.727 for other parties who might be involved in 00:25:09.747 --> 00:25:11.288 the evidence collection process. 00:25:12.028 --> 00:25:17.750 Just continuing along here with the tools 00:25:17.770 --> 00:25:18.531 that are available, 00:25:18.852 --> 00:25:21.032 we have also included a range of 00:25:21.113 --> 00:25:24.214 checklists for implementation, 00:25:24.335 --> 00:25:26.215 and I'm just going to quickly grab a 00:25:26.276 --> 00:25:27.916 couple of these and demonstrate. 00:25:28.317 --> 00:25:28.916 The first one, 00:25:29.136 --> 00:25:31.778 Responsible AI Adoption and Implementation 00:25:31.798 --> 00:25:32.239 Checklist. 00:25:32.739 --> 00:25:37.261 So this is a more extensive version of 00:25:37.521 --> 00:25:42.544 the guidance for AI adoption from the 00:25:42.584 --> 00:25:43.724 National AI Centre. 00:25:44.224 --> 00:25:46.547 So I showed you before an implementation 00:25:46.567 --> 00:25:48.627 checklist that was six pages that had the 00:25:48.768 --> 00:25:49.428 key information. 00:25:49.887 --> 00:25:51.869 This one is a much more detailed 00:25:51.950 --> 00:25:54.131 implementation check if you're doing a 00:25:54.211 --> 00:25:57.413 high risk or a wide scale adoption of 00:25:57.593 --> 00:25:59.233 AI for a particular function. 00:26:00.294 --> 00:26:01.074 So as you can see, 00:26:01.575 --> 00:26:03.994 it goes up to twenty four pages. 00:26:04.355 --> 00:26:06.836 It's still got those same essential areas. 00:26:06.875 --> 00:26:09.037 So it starts off with area one, 00:26:09.336 --> 00:26:10.596 decide who is accountable. 00:26:11.037 --> 00:26:12.958 And then it's got much more detailed 00:26:13.218 --> 00:26:15.818 information and checklists on the 00:26:16.179 --> 00:26:18.339 implementation actions and other 00:26:18.380 --> 00:26:19.819 information that you need to be putting in 00:26:19.859 --> 00:26:22.560 place within your organisation for, again, 00:26:22.580 --> 00:26:24.922 those higher risks or those wider scale. 00:26:25.721 --> 00:26:27.844 I'm not going to go through all of 00:26:27.864 --> 00:26:28.104 that, 00:26:28.545 --> 00:26:30.006 but just to highlight that it's there. 00:26:30.046 --> 00:26:30.365 Again, 00:26:30.746 --> 00:26:34.189 it's an excellent internal auditing tool 00:26:34.589 --> 00:26:36.891 if you want to internally audit one of 00:26:36.912 --> 00:26:40.374 your AI implementations against the 00:26:40.433 --> 00:26:41.895 guidance for AI adoption, 00:26:41.935 --> 00:26:44.096 the new national guidance for what we need 00:26:44.136 --> 00:26:44.597 to be doing. 00:26:45.097 --> 00:26:45.518 So, again, 00:26:45.557 --> 00:26:48.019 just connecting those two together, 00:26:48.039 --> 00:26:49.840 we've got the shorter implementation 00:26:49.861 --> 00:26:52.563 checklist for smaller AI implementations 00:26:52.603 --> 00:26:54.003 and perhaps for smaller providers, 00:26:54.483 --> 00:26:56.025 and then we've got that extensive, 00:26:56.265 --> 00:26:58.507 twenty-four-page implementation check that 00:26:58.527 --> 00:27:02.009 covers the full guidance from the National 00:27:02.088 --> 00:27:05.171 AI Centre for robust AI implementations. 00:27:05.810 --> 00:27:07.132 some of the other checklists we've got 00:27:07.192 --> 00:27:11.855 here we have also provided a checklist on 00:27:12.154 --> 00:27:14.537 ai generated content and this is directly 00:27:14.557 --> 00:27:16.499 aligned to the new guidance released by 00:27:16.538 --> 00:27:18.920 the national ai center the guide for 00:27:18.940 --> 00:27:20.981 business on what we need to be doing 00:27:21.142 --> 00:27:24.084 for transparency measures when we're using 00:27:24.304 --> 00:27:25.545 ai generated content 00:27:26.005 --> 00:27:27.685 And you can see this checklist is broken 00:27:27.726 --> 00:27:28.646 into three areas. 00:27:28.727 --> 00:27:31.308 They're system deployers or users of AI, 00:27:31.749 --> 00:27:33.730 which are obviously our users in our 00:27:33.770 --> 00:27:34.571 organizations. 00:27:34.932 --> 00:27:36.712 There's some for system developers where 00:27:36.732 --> 00:27:39.734 you're building AI into your applications 00:27:40.135 --> 00:27:42.758 and you might have application suppliers, 00:27:42.897 --> 00:27:44.759 you might have your tech suppliers who are 00:27:44.798 --> 00:27:45.319 doing that, 00:27:45.539 --> 00:27:46.681 or you might be building 00:27:47.644 --> 00:27:49.307 particular applications within your 00:27:49.347 --> 00:27:51.791 organisation where you're adding AI and 00:27:51.811 --> 00:27:52.834 you're a system developer. 00:27:53.174 --> 00:27:54.436 And then there's also a checklist for 00:27:54.477 --> 00:27:55.839 model developers, 00:27:55.880 --> 00:27:57.542 which is obviously thinking about the 00:27:57.583 --> 00:28:00.728 likes of OpenAI and Anthropic, et cetera. 00:28:01.666 --> 00:28:04.789 And the checklist here for system 00:28:04.809 --> 00:28:06.070 deployers, for users, 00:28:06.750 --> 00:28:08.913 what we need to have in place if 00:28:08.952 --> 00:28:11.394 we're using AI-generated content for 00:28:11.414 --> 00:28:13.676 system developers, and again, 00:28:13.797 --> 00:28:14.978 for model developers. 00:28:15.518 --> 00:28:16.838 And you can see the transparency 00:28:16.878 --> 00:28:17.759 mechanisms here. 00:28:17.819 --> 00:28:19.662 There's a three-step process. 00:28:20.201 --> 00:28:22.903 to work out what kinds of transparency 00:28:22.943 --> 00:28:24.244 mechanisms you need to use. 00:28:24.325 --> 00:28:26.866 Do you need to label your AI-generated 00:28:26.906 --> 00:28:27.307 content? 00:28:27.508 --> 00:28:29.429 Do you need to use a watermark on 00:28:29.489 --> 00:28:31.369 any images or videos that you might have 00:28:31.410 --> 00:28:32.490 created with AI? 00:28:32.932 --> 00:28:33.711 Or indeed, 00:28:33.791 --> 00:28:36.193 what metadata do you need to be storing 00:28:36.534 --> 00:28:39.777 on your AI-generated content? 00:28:40.317 --> 00:28:41.357 So we've got a bit of a risk 00:28:41.397 --> 00:28:42.098 assessment there. 00:28:42.138 --> 00:28:43.259 Again, first of all, 00:28:44.240 --> 00:28:46.903 isn't this risk based approach to our 00:28:46.943 --> 00:28:49.807 operations just so evident now as expected 00:28:49.826 --> 00:28:51.368 in the twenty twenty five standards for 00:28:51.409 --> 00:28:51.970 RTOs? 00:28:52.450 --> 00:28:53.832 So we're doing a risk assessment. 00:28:54.172 --> 00:28:54.952 Then step two, 00:28:54.992 --> 00:28:57.996 we're looking at what involvement level 00:28:58.036 --> 00:28:58.876 was there with AI. 00:29:00.430 --> 00:29:02.692 So we can see here the automation, 00:29:02.751 --> 00:29:03.731 human oversight, 00:29:04.173 --> 00:29:07.694 the extent of AI generated or modified 00:29:07.755 --> 00:29:08.236 content, 00:29:08.635 --> 00:29:10.557 and then the potential for AI to have 00:29:10.616 --> 00:29:13.318 changed the meaning of content as part of 00:29:13.338 --> 00:29:14.579 that generated content. 00:29:15.181 --> 00:29:16.882 And then step three is to select the 00:29:16.942 --> 00:29:18.303 transparency mechanism. 00:29:18.742 --> 00:29:20.605 And we've got some examples here as well 00:29:20.964 --> 00:29:23.468 in a vet provider context when you may 00:29:23.488 --> 00:29:25.710 or may not be using labelling of the 00:29:25.849 --> 00:29:27.152 AI generated content, 00:29:27.551 --> 00:29:29.273 when you may or may not be required 00:29:29.314 --> 00:29:30.454 to store the metadata, 00:29:30.815 --> 00:29:32.336 and when you may or may not be 00:29:32.436 --> 00:29:34.839 using watermarking of that content. 00:29:35.160 --> 00:29:36.621 So it's quite an extensive list that 00:29:36.661 --> 00:29:38.462 covers the full guidance that's been 00:29:38.502 --> 00:29:38.923 provided. 00:29:40.932 --> 00:29:44.156 If I then go to another checklist that 00:29:44.196 --> 00:29:45.198 we've included, 00:29:45.258 --> 00:29:47.540 the use of AI checklists here at the 00:29:47.560 --> 00:29:49.123 bottom, I'll just quickly mention as well. 00:29:49.683 --> 00:29:51.326 These are shorter checklists that are 00:29:51.346 --> 00:29:52.968 being used in the flow of work. 00:29:52.988 --> 00:29:55.171 So once you've got all of those other 00:29:55.851 --> 00:29:57.313 arrangements in place and you've 00:29:57.353 --> 00:29:59.395 effectively implemented your AI system, 00:29:59.816 --> 00:30:01.156 These are the checklists that are being 00:30:01.317 --> 00:30:03.599 used by your users as they're using AI 00:30:04.180 --> 00:30:07.163 or are being used by your monitoring 00:30:07.202 --> 00:30:09.025 process where you're monitoring what your 00:30:09.085 --> 00:30:10.906 personnel or indeed what your students are 00:30:10.926 --> 00:30:11.707 doing with AI. 00:30:12.107 --> 00:30:12.888 In the first instance, 00:30:12.909 --> 00:30:15.090 we've included a checklist before you act. 00:30:15.490 --> 00:30:18.374 Before you implement anything with AI as 00:30:18.973 --> 00:30:21.155 an individual within the organisation, 00:30:21.396 --> 00:30:23.278 just quickly confirm that all of these 00:30:23.317 --> 00:30:24.660 things have been covered off. 00:30:25.220 --> 00:30:27.343 we've also then included the transparency 00:30:27.403 --> 00:30:29.606 mechanism so if you've just generated 00:30:29.666 --> 00:30:31.990 content from ai as a per as a 00:30:32.029 --> 00:30:34.594 staff member personnel or as a student 00:30:35.134 --> 00:30:37.498 this could be used for as well then 00:30:37.597 --> 00:30:39.119 what kind of labeling do i need to 00:30:39.140 --> 00:30:39.520 be doing 00:30:41.695 --> 00:30:44.299 for that particular AI implementation. 00:30:44.880 --> 00:30:45.361 So again, 00:30:45.381 --> 00:30:47.743 you can see there sort of two checklists 00:30:47.923 --> 00:30:50.907 in the use of AI that's designed to 00:30:50.929 --> 00:30:54.053 be used every time AI is being used 00:30:54.093 --> 00:30:57.057 to generate content for a particular 00:30:57.156 --> 00:30:58.979 activity within the organisation. 00:30:59.660 --> 00:31:01.861 So hopefully you found that really useful. 00:31:01.881 --> 00:31:03.921 You can see some really extensive tools 00:31:03.961 --> 00:31:05.241 there for implementation, 00:31:05.662 --> 00:31:07.781 whether it be implementation checklists, 00:31:08.123 --> 00:31:10.343 whether it be example inclusions in key 00:31:10.383 --> 00:31:12.963 parts of your quality management system 00:31:12.983 --> 00:31:14.544 within your provider, privacy, 00:31:14.943 --> 00:31:17.345 application for enrolment, assessment, 00:31:17.545 --> 00:31:18.065 et cetera, 00:31:18.506 --> 00:31:21.846 and then some key checklists that you 00:31:21.886 --> 00:31:24.606 should be using when you're using AI on 00:31:24.626 --> 00:31:26.208 an ongoing basis for projects. 00:31:27.188 --> 00:31:28.828 The last section I'll just quickly mention 00:31:28.848 --> 00:31:30.549 here as well is in the monitoring and 00:31:30.569 --> 00:31:31.190 evaluation. 00:31:31.210 --> 00:31:31.911 Now, as I mentioned, 00:31:31.931 --> 00:31:33.951 you've already seen checklists that can be 00:31:33.971 --> 00:31:35.172 getting used for monitoring. 00:31:35.492 --> 00:31:37.834 You've already seen our stakeholder survey 00:31:37.874 --> 00:31:39.414 that you can be using for collating 00:31:39.775 --> 00:31:41.635 feedback as part of your monitoring. 00:31:42.056 --> 00:31:43.616 We've got two other tools here. 00:31:44.076 --> 00:31:46.298 One is the privacy checklist for the use 00:31:46.338 --> 00:31:46.558 of AI. 00:31:48.499 --> 00:31:48.959 And again, 00:31:49.118 --> 00:31:51.980 this is something that you can use 00:31:52.340 --> 00:31:55.721 pre-confirmation when you're preparing 00:31:55.862 --> 00:31:59.884 your AI and it's for preparing to 00:32:00.003 --> 00:32:01.285 implement an AI tool. 00:32:01.704 --> 00:32:03.086 And you can see the referencing there 00:32:03.145 --> 00:32:05.527 again to those three analyses that what do 00:32:05.567 --> 00:32:07.788 our stakeholder impact and then the 00:32:07.807 --> 00:32:09.568 privacy impact and then the risk 00:32:09.608 --> 00:32:12.190 assessment in order to make a judgment for 00:32:12.210 --> 00:32:13.411 the implementation of this. 00:32:14.151 --> 00:32:17.653 And then we've got the guidance from the 00:32:18.075 --> 00:32:19.336 Office of the Australian Information 00:32:19.355 --> 00:32:23.138 Commissioner around the use of AI content 00:32:23.179 --> 00:32:24.601 from the privacy context. 00:32:25.000 --> 00:32:25.501 So again, 00:32:25.541 --> 00:32:27.143 it's a straightforward checklist. 00:32:27.403 --> 00:32:29.845 If you're about to use any personal or 00:32:29.865 --> 00:32:30.925 sensitive information, 00:32:31.666 --> 00:32:33.788 in any AI tool, very, 00:32:33.827 --> 00:32:36.807 very useful for you to be undertaking this 00:32:36.847 --> 00:32:37.807 privacy checklist. 00:32:38.269 --> 00:32:40.989 And if you're monitoring usage or indeed 00:32:41.009 --> 00:32:43.150 doing that formal evaluation of usage, 00:32:43.230 --> 00:32:43.630 then again, 00:32:43.650 --> 00:32:45.869 that can be a really handy audit tool. 00:32:46.390 --> 00:32:47.590 And speaking of audit tools, 00:32:47.871 --> 00:32:49.770 that is the final tool that we've included 00:32:49.810 --> 00:32:50.230 in the pack. 00:32:50.250 --> 00:32:52.251 There's over twenty resource tools and 00:32:52.271 --> 00:32:53.332 templates in this pack. 00:32:53.872 --> 00:32:57.315 And the last one is our internal audit 00:32:57.355 --> 00:32:58.576 tool for AI adoption. 00:32:59.017 --> 00:33:01.838 We do recommend that you audit your AI 00:33:01.959 --> 00:33:03.319 usage on an annual basis. 00:33:03.700 --> 00:33:05.701 It is going to be across all of 00:33:05.721 --> 00:33:07.303 your operating functions within your 00:33:07.343 --> 00:33:08.044 organisation, 00:33:08.064 --> 00:33:10.385 but you do need to be demonstrating that 00:33:10.425 --> 00:33:13.067 you are reviewing and confirming that your 00:33:13.087 --> 00:33:15.509 AI usage is compliant on an ongoing basis. 00:33:15.871 --> 00:33:17.751 You can see in the audit criteria here, 00:33:17.811 --> 00:33:19.693 this particular audit tool has been 00:33:19.733 --> 00:33:20.595 designed against 00:33:21.174 --> 00:33:24.718 the six implementation practices from our 00:33:24.778 --> 00:33:26.199 guidance for AI adoption. 00:33:26.640 --> 00:33:29.122 It's also aligned to the privacy 00:33:29.162 --> 00:33:30.643 arrangements from the Office of the 00:33:30.663 --> 00:33:31.964 Australian Information Commissioner, 00:33:32.424 --> 00:33:34.105 and it's also aligned to the twenty twenty 00:33:34.145 --> 00:33:35.488 five standards for RTOs. 00:33:35.567 --> 00:33:36.268 Four point one, 00:33:36.588 --> 00:33:37.930 looking at our governance of the 00:33:37.970 --> 00:33:38.750 organisation, 00:33:39.111 --> 00:33:41.633 including all of that governance required 00:33:41.653 --> 00:33:45.355 for effective AI implementation and that 00:33:45.455 --> 00:33:47.637 risk management from outcome standard four 00:33:47.657 --> 00:33:48.159 point three. 00:33:48.578 --> 00:33:50.942 So I've got the standard internal auditing 00:33:50.961 --> 00:33:51.522 tool here. 00:33:51.542 --> 00:33:52.584 We've got the audit plan. 00:33:53.384 --> 00:33:54.666 We've got the sampling plan. 00:33:54.807 --> 00:33:56.949 What parts of the organisation, 00:33:57.009 --> 00:33:59.071 what AI implementations are you going to 00:33:59.112 --> 00:33:59.531 sample? 00:34:00.113 --> 00:34:01.734 We've got our effectiveness plan. 00:34:05.846 --> 00:34:07.126 So we're looking at our quality 00:34:07.166 --> 00:34:07.826 objectives, 00:34:07.906 --> 00:34:09.768 our performance and outcome indicators and 00:34:09.807 --> 00:34:10.789 targets we've set, 00:34:11.208 --> 00:34:12.088 and then we're auditing, 00:34:12.349 --> 00:34:14.110 are we achieving what we said we'd 00:34:14.130 --> 00:34:14.449 achieve? 00:34:14.869 --> 00:34:16.990 Is our performance what we expected it to 00:34:17.030 --> 00:34:17.210 be? 00:34:17.731 --> 00:34:19.972 Are our outcomes what we expected it to 00:34:19.992 --> 00:34:20.233 be? 00:34:20.512 --> 00:34:22.333 Systematic monitoring and evaluation, 00:34:22.713 --> 00:34:24.954 critical examination for self-assurance. 00:34:25.295 --> 00:34:27.436 And then underneath that quality audit, 00:34:27.815 --> 00:34:29.737 we've also got the compliance audit. 00:34:29.757 --> 00:34:33.778 So we've included the privacy guidelines, 00:34:34.338 --> 00:34:36.922 and an auditing tool are very similar to 00:34:36.963 --> 00:34:39.226 that checklist for that audit. 00:34:39.766 --> 00:34:42.469 We've included the guidelines for AI 00:34:42.510 --> 00:34:45.293 generated content and an audit tool there. 00:34:45.353 --> 00:34:47.556 Do we have this in place for all 00:34:47.836 --> 00:34:51.081 of our AI usage where we are generating 00:34:51.121 --> 00:34:51.561 content? 00:34:53.023 --> 00:34:57.025 And we then have the responsible AI 00:34:57.045 --> 00:34:58.385 adoption checklist. 00:34:58.746 --> 00:35:00.146 And this is the full tool. 00:35:00.206 --> 00:35:01.547 So if you are going to do an 00:35:01.606 --> 00:35:04.588 internal audit against the full guidance 00:35:04.608 --> 00:35:07.289 for AI adoption from the National AI 00:35:07.329 --> 00:35:07.670 Centre, 00:35:07.710 --> 00:35:09.771 then we've got that full audit tool there. 00:35:10.331 --> 00:35:12.012 I'm going to just run right through that 00:35:12.452 --> 00:35:13.231 towards the bottom. 00:35:13.512 --> 00:35:15.393 So that's the full twenty four pages there 00:35:15.432 --> 00:35:16.474 to allow you to audit it. 00:35:17.034 --> 00:35:18.956 And then we have the standards for RTO's 00:35:18.996 --> 00:35:21.240 twenty twenty five format as well to give 00:35:21.260 --> 00:35:23.043 you the format to be able to audit 00:35:23.304 --> 00:35:25.447 your governance approach to AI under 00:35:25.628 --> 00:35:27.851 outcome standard four point one and your 00:35:27.911 --> 00:35:30.094 risk management approach to AI under 00:35:30.255 --> 00:35:31.797 outcome standard four point three. 00:35:33.208 --> 00:35:35.429 So very, very extensive audit tool. 00:35:35.528 --> 00:35:37.730 You can cut and paste that audit tool 00:35:37.789 --> 00:35:39.449 if you just want to audit the privacy 00:35:39.469 --> 00:35:40.070 arrangements, 00:35:40.469 --> 00:35:41.931 if you just want to audit the twenty 00:35:41.951 --> 00:35:43.431 twenty five standards requirements, 00:35:43.831 --> 00:35:46.132 if you just want to audit the National 00:35:46.632 --> 00:35:47.572 Center requirements, 00:35:47.652 --> 00:35:49.793 if you just want to audit quality 00:35:50.494 --> 00:35:52.815 requirements and the performance and 00:35:52.875 --> 00:35:55.054 outcome measures that you've set for the 00:35:55.275 --> 00:35:55.956 organization. 00:35:56.735 --> 00:35:57.335 So as I mentioned, 00:35:57.356 --> 00:36:00.737 the extensive pack now for their providers 00:36:00.777 --> 00:36:02.478 to make sure that we can go back, 00:36:02.719 --> 00:36:04.918 we've all been doing lots of bits and 00:36:04.958 --> 00:36:09.161 pieces with AI in the last three years. 00:36:09.621 --> 00:36:11.922 Now is the time we've got the definitive 00:36:11.942 --> 00:36:13.202 guidance that's come out 00:36:13.682 --> 00:36:15.264 from the Office of the Australian 00:36:15.284 --> 00:36:16.304 Information Commissioner, 00:36:16.704 --> 00:36:18.545 from the National AI Centre. 00:36:18.864 --> 00:36:20.626 We've got the Twenty Twenty Five Standards 00:36:20.666 --> 00:36:21.226 for RTOs. 00:36:21.246 --> 00:36:22.927 These are all compliance requirements and 00:36:22.987 --> 00:36:25.648 expectations that we need to meet in our 00:36:25.728 --> 00:36:27.108 implementation of AI. 00:36:27.429 --> 00:36:29.429 So now that we have those confirmed 00:36:29.469 --> 00:36:30.090 requirements, 00:36:30.429 --> 00:36:32.610 it's time to step back and make sure 00:36:32.630 --> 00:36:33.871 that everything that we've been doing for 00:36:33.891 --> 00:36:35.791 the last three years with AI that we 00:36:35.831 --> 00:36:36.693 have implemented 00:36:37.413 --> 00:36:39.675 is meeting these requirements moving 00:36:39.715 --> 00:36:42.378 forward and for new AI implementations 00:36:42.418 --> 00:36:44.358 that we have the governance and the 00:36:44.398 --> 00:36:47.262 implementation structure in place to get 00:36:47.302 --> 00:36:50.123 that implementation right for responsible 00:36:50.304 --> 00:36:52.606 use of AI and that we've got the 00:36:52.646 --> 00:36:54.788 monitoring and evaluation processes in 00:36:54.849 --> 00:36:58.231 place as well to confirm that it remains 00:36:58.271 --> 00:36:58.911 effective, 00:36:59.251 --> 00:37:00.793 that our use of AI is on a 00:37:00.813 --> 00:37:01.614 quality basis, 00:37:01.795 --> 00:37:04.177 on an ongoing basis within our operations. 00:37:04.717 --> 00:37:07.338 So some key tips here to finish. 00:37:07.697 --> 00:37:08.858 We need to understand all of the 00:37:08.898 --> 00:37:09.557 requirements. 00:37:10.018 --> 00:37:11.298 All of these requirements, 00:37:11.338 --> 00:37:12.358 some of them are very new, 00:37:12.539 --> 00:37:13.340 released in late, 00:37:13.380 --> 00:37:16.420 they're now in place and we need to 00:37:16.440 --> 00:37:18.280 make sure that we've got these 00:37:18.380 --> 00:37:20.362 arrangements embedded within our 00:37:20.422 --> 00:37:21.282 organisations. 00:37:21.322 --> 00:37:23.262 We do need to approach the use of 00:37:23.402 --> 00:37:25.103 AI in a responsible manner. 00:37:26.184 --> 00:37:27.985 Every organisation should be doing that, 00:37:28.264 --> 00:37:30.927 especially education institutions being 00:37:30.967 --> 00:37:32.588 their providers should be doing that. 00:37:32.789 --> 00:37:35.391 And we do need to justify our use 00:37:35.451 --> 00:37:37.291 case for AI in each case. 00:37:37.713 --> 00:37:40.355 What is that impact on our stakeholders? 00:37:40.815 --> 00:37:42.936 What privacy impact have we undertaken to 00:37:42.956 --> 00:37:44.818 make sure that we have got our privacy 00:37:44.838 --> 00:37:45.699 arrangements covered? 00:37:46.059 --> 00:37:48.541 What risk assessment have we put in place 00:37:48.561 --> 00:37:50.663 to make sure that we are managing and 00:37:51.123 --> 00:37:52.985 mitigating, identifying and managing? 00:37:53.565 --> 00:37:55.125 using the twenty twenty five standards 00:37:55.164 --> 00:37:55.565 language, 00:37:56.585 --> 00:37:58.626 the risks that are associated with the 00:37:58.666 --> 00:38:00.365 implementation of these tools. 00:38:00.686 --> 00:38:02.206 We need to have all those key processes 00:38:02.246 --> 00:38:02.786 in place. 00:38:03.146 --> 00:38:05.527 So we do need to have that information 00:38:05.567 --> 00:38:06.867 in place in privacy, 00:38:07.228 --> 00:38:08.487 application for enrolment, 00:38:08.768 --> 00:38:11.188 training delivery may require some 00:38:11.708 --> 00:38:12.268 guidance, 00:38:12.309 --> 00:38:14.989 certainly at the assessment delivery side 00:38:15.168 --> 00:38:15.588 as well. 00:38:15.929 --> 00:38:17.889 We need to be training all our personnel 00:38:18.050 --> 00:38:19.469 in all of these expectations. 00:38:20.010 --> 00:38:21.351 that they can't just be throwing 00:38:21.391 --> 00:38:23.253 information into AI, 00:38:23.333 --> 00:38:24.795 any AI tool that they want, 00:38:25.297 --> 00:38:26.958 that they do need to be following a 00:38:27.018 --> 00:38:29.722 robust process and they do particularly 00:38:29.762 --> 00:38:31.425 need to be watching out for personal 00:38:31.465 --> 00:38:33.527 information, student names, for example, 00:38:34.307 --> 00:38:36.610 and sensitive information and what can... 00:38:37.492 --> 00:38:39.713 and cannot be done with that information 00:38:39.813 --> 00:38:42.835 based on your policy and procedures 00:38:42.875 --> 00:38:45.076 framework that you put in place for them 00:38:45.135 --> 00:38:45.536 to do. 00:38:45.976 --> 00:38:46.976 And then we need to be tracking, 00:38:47.376 --> 00:38:51.099 reviewing and updating that use of AI 00:38:51.159 --> 00:38:53.440 monitoring on an ongoing basis and then 00:38:53.481 --> 00:38:55.822 doing that periodic evaluation, 00:38:56.021 --> 00:38:56.782 internal audit, 00:38:57.163 --> 00:38:59.244 critical examination for self-assurance 00:38:59.364 --> 00:39:02.085 and making sure that our AI usage is 00:39:02.146 --> 00:39:02.545 meeting 00:39:03.106 --> 00:39:06.347 the responsible guidelines that we expect 00:39:06.487 --> 00:39:07.788 now within our community. 00:39:08.208 --> 00:39:11.389 I hope you found the pack really useful 00:39:11.730 --> 00:39:13.831 in helping you to easily understand and 00:39:13.871 --> 00:39:15.931 implement those arrangements within your 00:39:15.992 --> 00:39:16.612 operations.