Go to top of page

TEQSA Stakeholder Survey 2018: Report of Overall Findings

24 December 2018

Executive summary

In June 2018 the Tertiary Education Quality Standards Agency (TEQSA) conducted the third of its sector wide surveys. The purpose of the survey was to increase TEQSA’s accountability, better understand its impact on higher education providers, and improve its performance. 

TEQSA engaged Australian Survey Research (ASR) to deploy and analyse a web survey of 235 principal contacts (PC) and 42 peak, professional and student bodies (PPSB). A total of 156 principal contacts and 24 PPSBs completed the survey. 

The 2018 survey consisted of:

  • One version for principal contacts of all higher education providers (those registered as well as those who had submitted initial registration applications)
  • A brief version for the operational head of selected peak, professional and student bodies.

The 2018 survey items were almost identical to those used in 2017. The content of the surveys focused on TEQSA’s key performance indicators from its current Regulator Performance Framework. In addition, principal contacts were asked about various recent interactions with TEQSA including applications, case managers and roundtables. All respondents in all surveys were asked to rate TEQSA’s overall performance as a regulator.

Response

There were sufficient numbers of principal contact responses and a well-constructed sample to conclude that the PC survey was representative of the population at the 95% confidence interval and a confidence level of ±4.6%. The PPSB survey had a ±13% confidence interval meaning that results from that particular survey could, at best, be interpreted as indicative only.

Key findings

Results were analysed to produce top 2 scores (the proportion of respondents selecting the two most positive rating points—excellent and good). Don’t know / not applicable and no answer responses were excluded from all top 2 score calculations. Top 2 scores of around 80% and above are considered a good result in customer satisfaction research.

PC survey: 2018 highs and lows

The table below shows the 2018 survey items which achieved a top 2 score of 80% and over (when rounded). Guidance and support materials, the TEQSA conference and the quality and relevance of information were highly rated.
 

PC: TOP SCORING ITEMS

2018
n

2018
TOP 2 SCORE (%)

2017
TOP 2 SCORE (%)*

Conference: Opportunity to interact with other delegates

97

90.7

91.7

Guidance & Support materials: Relevance of information

138

90.6

92.9

Guidance & Support materials: The quality of the information

138

89.1

94.1

Guidance & Support materials: Ease of access to that information

138

87.7

82.4

Guidance & Support materials: Usefulness of information

138

86.2

90.6

Conference: The quality of speakers

97

85.6

90.4

Conference: The program

97

83.5

89.2

Guidance & Support materials: Amount of information

137

83.2

81.2

Conference: Relevance of material presented

97

82.5

86.7

KPI 2: Relevance of information

154

82.5

87.1

KPI 2: Quality of regulatory information

154

80.5

88.6

The lower scoring items (below 60% top 2 score) in the PC survey are displayed in the table immediately below. These issues were all reflected in respondent comments, particularly around timeliness, stability in case management, sector engagement and requests for some or more visits or face-to-face meetings.

The key areas requiring improvements are TEQSA’s streamlining, taking action that are proportionate to risks, case management (organisational knowledge and consideration of needs), and building relationships through engagement and visits. A view still exists with private / independent providers that TEQSA is university centric (and using the term non-university is an example) and does not understand the particular situations and offerings of non-public providers: that TEQSA is hostile to them.

TEQSA’s case management approach is well-received. Where it works well it is lauded; where it doesn’t it is a major source of concern. While considerable effort has been made with provider engagement, providers are hungry for much more, particularly personalised engagement.

Regulatory burden is felt by many, irrespective of type or size of provider.

Statistically significantly different (p<0.05) results highlighted in bold.

PC: LOWER SCORING ITEMS

2018
n

2018 TOP 2 SCORE (%)

2017 TOP 2 SCORE (%)

KPI 5: Consistency of information

152

58.6

67.6

KPI 3: Consultative approach

132

56.1

53.2

KPI 2: Completeness of information

123

55.3

71.2

KPI 4: Reuse of material

102

53.9

56.4

KPI 6: Engagement

154

53.9

58.8

KPI 5: Consistency of decisions

117

53.8

60.6

KPI 2: Timeliness of information after making a decision

113

52.2

54.5

KPI 6: Making process improvements

138

52.2

61.8

Case mgt approach: Consideration of your needs

142

51.4

55.4

KPI 2: Communicating streamlining initiatives

144

51.4

59.2

KPI 4: Timely coordination of visits

77

49.4

53.3

Case mgt approach: Knowledge of your organisation

145

48.3

52.2

KPI 3: Actions proportionate to risks

116

47.4

60.9

KPI 1: Streamlining to reduce burden

138

47.1

59.2

PC survey: overall rating

Principal contacts rating of TEQSA’s overall performance as a regulator has remained above a 70% top 2 score (71.1%), but has declined since 2017 (79.7%) and 2016 (82.3%). This trend does not differ significantly by provider attribute (type, size, category, etc).

PC survey: attribute differences

Responses from the principal contact survey were analysed by various provider attributes to understand where there were similarities and differences between provider sub-groups. This analysis helps identify issues with particular segments and allows TEQSA to tailor initiatives to these particular groups.

Provider size showed no statistically significant differences—an unusual result for this type of survey and a good result for TEQSA and one that has remained over time. It means that, from a provider perspective, TEQSA does not treat small, medium or large providers differently on the items surveyed. Results for state (location) were also not significantly different.

There were, however, some differences in the following sub-groups:

  • Self-accrediting authority (SAA) or not
  • High / moderate financial position risk as assessed by TEQSA
  • High / moderate student risk as assessed by TEQSA
  • Registered Training Organisation (RTO) activity
  • CRICOS / not CRICOS registered
  • Category
  • Market groupings.

Providers with some or part SAA had the largest number of differences and were considerably more positive than providers without this authority. All or part SAA providers rated TEQSA’s overall performance as a regulator with an 88.6% top 2 score while non-SAA providers rated TEQSA overall at 61.8%.

Survey comparison

For this piece of analysis, a simple average of the top 2 scores for each item was calculated within a topic (KPI) of the principal contact survey. These six average top 2 scores were then compared with the top 2 scores of the same KPIs within the PPSB survey, along with the top 2 scores of TEQSA’s overall rating as a regulator.

The comparative results are displayed in the table below. It shows that PPSBs were considerably more positive than providers about TEQSA’s performance on all KPIs and TEQSA’s performance. We note that there was a very small sample size in the PPSB survey which means that results must be treated as indicative only.

KPI / ITEM and TOP 2 SCORE (%) FOR EACH SURVEY

PRINCIPAL CONTACT
n=156

PEAK / PROF / STUDENT BODY
n=24

KPI 1 - Regulation does not impede efficient operation

55.8

93.7

KPI 2 - TEQSA’s communication with your organisation

63.8

87.1

KPI 3 - Regulatory actions are proportionate to risks

51.7

93.3

KPI 4 - TEQSA has a streamlined and co-ordinated approach

51.6

80.0

KPI 5 - TEQSA is open, transparent and consistent in its dealings

61.5

91.3

KPI 6 - TEQSA continues to improve its regulatory framework

57.2

79.0

Overall TEQSA's performance as a regulator over the last 12 months

71.1

100.0

Conclusions

For TEQSA’s third survey of stakeholders, there were some excellent results, balanced by many results that declined universally in the last 12 months. Sub-groups within the provider population had some different views on aspects of TEQSA’s performance and these differences need to be recognised. There were some providers (usually low risk and self-accrediting) who were happy with many aspects TEQSA’s performance while for profit and prospective providers were less positive about particular aspects of TEQSA’s performance, including overall performance.

The 2018 results clearly indicate where providers thought TEQSA was doing well (its conference and the quality and relevance of guidance materials and regulatory information) and where it could improve (streamlining, speed of response, consultation and case management for all and CRICOS applications where relevant).

While TEQSA’s case management approach is welcomed, all providers want greater consideration of their needs and an opportunity to build ongoing relationships with TEQSA.

Importantly, overall TEQSA was fairly well-regarded by providers and extremely well-regarded by peak bodies as a regulator assuring the quality of Australia’s higher education.

Introduction

In June 2018 TEQSA conducted sector-wide stakeholder surveys for 2017-18. The purpose of the surveys was to increase TEQSA’s accountability, better understand its impact on higher education providers, and improve its performance. Results were also used in annual reporting activities. The feedback activity consisted of two surveys:

  • A provider-specific version with principal contacts for all higher education providers (those registered as well as those who had submitted initial registration applications)
  • A brief survey for the operational head of selected peak, professional and student bodies.

TEQSA engaged Australian Survey Research (ASR) to help design, deploy, analyse and report on the surveys. 

The report outlines the methodology used to conduct and analyse the web surveys as well as key findings from each survey including provider attribute differences. The questionnaires used in the surveys form an appendix to the report.

Methodology

This section outlines how the two surveys were developed; how survey participants were identified; how the survey was administered and analysed; and the composition of the response sample.

Questionnaire development

In 2016 TEQSA and ASR developed a series of questions based around TEQSA’s Regulator Performance Framework (2015-16) which encompassed six key performance indicators (KPIs). The Framework also incorporated the key strategies and metrics within the TEQSA Corporate Plan 2015-19. 

The questionnaire developed for TEQSA’s principal contacts (PC) had an operational focus and included questions around recent interactions between themselves and TEQSA. VC / CEOS were given an opportunity to comment separately within the principal contact survey. The peak and professional body (PPSB) survey was very short and focused on TEQSA’s overall KPI achievement.

The PC survey was pilot tested in 2016 with a small group of participants who were willing to be involved in the pilot phase. The pilot test resulted in a number of changes to the questionnaires. No pilot testing was conducted for the 2017 or 2018 surveys as questions and deployment method changed minimally. 

In 2017 some new questions were added about TEQSA’s conference and guidance and support materials. The 2018 surveys remained very similar to those used in 2017, with the addition of three questions around perceived quality or reputation of the Australian higher education sector.

Data collection

The two web questionnaires were loaded into ASR’s proprietary web surveying tool, SurveyManager and hosted on ASR’s internet servers located at a high security data centre in Melbourne’s CBD. 

TEQSA provided ASR with a full listing of all current and selected pending higher education providers (N=235) that it regulates or is likely to regulate across Australia. The lists included contact details of TEQSA’s principal contact within the provider. The listing also included provider attributes such as state, size, self-accrediting authority, etc, which were used to analyse responses. Results of this analysis are discussed later in the report. Further to the higher education provider list, TEQSA provided ASR with a list of peak, professional and student bodies’ names and contact details (N=42). 

Prior to going live with the full survey, TEQSA’s Chief Commissioner, Professor Nick Saunders, AO and its CEO Mr Anthony McClaran emailed a joint letter to the CEOs of all potential participants advising them of the survey and requesting their participation. Soon after, ASR sent invitation emails to the principal contact within each provider and each PPSB representative. The invitation email contained a unique hyperlink to access a recipient’s questionnaire.

ASR monitored response rates and sent two targeted reminder emails to all non-responders in each survey. The survey was in field from 4 June to 20 June 2018.

Data analysis

Results were analysed to produce top 2 scores (the proportion of respondents selecting the two most positive rating points) and frequency distributions. A z test was used to determine any statistical differences between attribute sub-groups which included self-accrediting authority, 2018 financial risk rating, 2018 student risk rating, category, state, provider size, RTO activity, CRICOS registration and market groupings. All tests were reported at the p<0.05 level (95% confidence level). See the box below for further explanation of confidence levels and intervals.

Top 2 scores were calculated using only the number of respondents who chose a rating point answer. In other words don’t know, not applicable and no answers (blank) were excluded from statistical calculations. A top 2% score of 100% can be interpreted as all respondents who answered a particular question indicated that TEQSA was performing at a good or excellent level on a particular item.

In some tables the total may not always equal 100.0%. This is due to rounding and is not an error.

Response and sample profile

A total of 235 principal contacts were invited to participate in their survey. A total of 156 principal contacts responded to the survey, yielding a response rate of 66%. The sample is statistically representative of the principal contacts population at the 95% confidence level and a ±4.6% confidence interval. This is an acceptable scientific research confidence interval.

A total of 42 representatives from peak, professional and student bodies were invited to participate in their survey. Twenty-four representatives from these bodies answered the survey achieving a response rate of 57%. The results for this survey have a confidence interval of ±13%. With this higher confidence interval and a small sample of 24, we suggest treating the PPSB results with caution and as indicative only.

  • Representativeness of a sample is often assessed at a 95% confidence level (accuracy) and a ±5% confidence interval (precision).

    The confidence interval (also called margin of error) is the plus-or-minus figure usually reported in newspaper or television opinion poll results. For example, if you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be "sure" that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.

    The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level.

    When you put the confidence level and the confidence interval together, you can say that you are 95% sure that the true percentage of the population is between 43% and 51%. The wider the confidence interval you are willing to accept, the more certain you can be that the whole population answers would be within that range.

    For example, if you asked a sample of 1000 people in a city which TV channel they preferred watching, and 60% said Channel A, you can be very certain that between 40% and 80% of all the people in the city actually do prefer that channel, but you cannot be so sure that between 59% and 61% of the people in the city prefer the channel. Reference.

Population / sample comparison

The profiles of the provider population and the survey sample were compared by state and category to identify any over/under-representation in the principal contact response set. Both profiles had very similar proportions (see tables immediately below) meaning that the response set showed no non-response bias, that is, the sample closely reflected the population on each attribute. As a result, the response sample was considered closely representative of the population and no weighting was applied to the principal contact survey response set.

Note that throughout this report, the total sample n count varies slightly. This is not an error. One institution chose to answer additionally and from a group perspective, so was not included in the population figures and had no attributes associated with it.
 

STATE

PRINCIPAL CONTACTS
POPULATION

RESPONSE
SAMPLE

 

Freq

%

Freq

%

NSW

101

43.3

72

46.5

VIC

62

26.6

38

24.5

QLD

26

11.2

15

9.7

SA

19

8.2

12

7.7

WA

17

7.3

12

7.7

ACT

5

2.1

3

1.9

NT

2

0.9

2

1.3

TAS

1

0.4

1

.6

Total

233

100.0

155

100.0

 

CATEGORY

PRINCIPAL CONTACTS
POPULATION

RESPONSE
SAMPLE

 

Freq

%

Freq

%

University

42

17.9

37

22.7

Higher Education Provider (HEP)

124

53.0

90

58.7

Prospective Higher Education Provider (Prop HEP)

68

29.1

28

18.1

Total

234

100.0

155

100.0

Data file

ASR has supplied de-identified and randomised raw data files to TEQSA. The files also contained all de-identified verbatim comments for TEQSA’s further investigation.

Principal contact survey key findings

This section outlines the key findings from the principal contact (PC) survey. Respondents were asked to rate TEQSA’s performance on a number of items. Results are presented by topic, in the same order as presented to respondents in the online questionnaire. For most items, top 2 scores (see orange note below) are presented along with a charted frequency distribution. The most common themes within free text comments are covered throughout each topic. 

  • Important notes about scores and charts: A top 2 score is the total proportion of respondents selecting the two most positive rating points in their answer to a question. When calculating the proportion of respondents in this answer category, any respondents who answered with don't know, not applicable or no answer / have been excluded from the base of the calculation. 

    As a result the percentage of green (dark and light green) in the following charts may not always be equivalent to the top 2 score as presented in tables. The table figures will usually be slightly higher. This is because the chart percentages include don’t know, not applicable, and no answer proportions.

    The charts have been sorted by the proportion of positive responses and are presented in descending order. 

    When reading the charts, it is useful to look at the proportion of green (positive) and the proportion of orange/red (negative) responses. More green than other colours means that positive ratings outweigh negative ratings. A lot of orange and red indicates considerable room for improvement.

In order to fit tables and charts within a page, item names have been abbreviated. Refer to appendix A for a table of abbreviations.

Warning about averages and summary results

The survey results presented in this section are based on aggregate scores, that is, all respondents’ answers taken together. Within the target group and the resulting answers there is considerable variation in views. This means that an average or summary answer does not reflect any single sub-group. Averaging loses detail, and for this survey, a detailed view makes a difference. When reading the summary results, it may be useful to keep this in mind. The section on attribute analysis makes the differences much clearer.

Key performance indicators

PC KPI 1: Regulation by TEQSA does not unnecessarily impede the efficient operation of your organisation

Sixty-five percent of PCs who rated TEQSA’s performance on the item opportunity to give feedback. rated it as either good or excellent. The other item under KPI 1, streamlining to reduce burden, was rated lower at 47%, and was the lowest rated item for the PC survey. Note that ratings for the item streamlining to reduce burden were significantly lower than in the 2017 results. Refer to the table below.

PC: KPI 1  

TOP 2 SCORE (%)*

Opportunity to give feedback  n=144

64.6

Streamlining to reduce burden  n=138

47.1

*Excludes don’t know / not applicable / no answer responses

The following chart shows the frequency distribution of answers (proportion of respondents choosing a particular answer) for KPI 1 items.

PC: KPI 1

 KPI 1

PC KPI 2: TEQSA’s communication with your organisation is clear, targeted and effective

Two of the seven items under KPI 2 rated above 80% which is a noteworthy result for a stakeholder feedback survey. Relevance of information scored highest, with 83% of PCs rating this item as either good or excellent. Quality of regulatory information was the second highest scoring item with a top 2 score of 81%.  

The items relating to completeness of information (55%) rated significantly lower this year than in 2017.

PC: KPI 2

TOP 2 SCORES (%)*

Relevance of information  n=154

82.5

Quality of regulatory information  n=154

80.5

Reasonable opportunity to address matters  n=115

64.3

Clarity of information  n=122

60.7

Completeness of information  n=123

55.3

Timeliness of information after making a decision  n=113

52.2

Communicating streamlining initiatives  n=144

51.4

*Excludes don’t know / not applicable / no answer responses

In the chart below, note the relatively high proportion of don’t know / not applicable / no answers for some items relating to TEQSA’s communication. This suggests that not all respondents had the experience or information to provide a rating response for these items. This may not be negative, but simply reflect a lack of opportunity or need.

PC: KPI 2

 KPI 2

PC: KPI 3 - Regulatory actions undertaken by TEQSA for your organisation are proportionate to the risks being managed

Compared with all other KPIs, top 2 scores remained considerably lower for KPI 3 with scores of 56% and 47% for the two items. The item actions proportionate to risks was the second lowest rated item in the PC survey.

Referring to the table and chart below, note the relatively high proportion of don’t know / not applicable / no answer responses for both items, suggesting that not all PCs had the experience or information to provide a rating response. 
 

PC: KPI 3

TOP 2 SCORE (%)*

Consultative approach  n=132

56.1

Actions proportionate to risks  n=116

47.4

*Excludes don’t know / not applicable / no answer responses

PC: KPI 3

 KPI 3

PC KPI 4: TEQSA has a streamlined and coordinated approach to compliance and monitoring for your organisation

The ratings for both items within KPI 4 dropped slightly between 2017 and 2018. The item reuse of material was rated as good or excellent by 54% and timely coordination of visits by 49% (compared to 56% and 53% in 2017). 

There was a considerable proportion of don’t know / not applicable / no answers for these two items, particularly for timely coordination of visits (47%). It is likely these respondents had not experienced a TEQSA staff visit in the last 12 months and therefore were not able to provide a rating.

PC: KPI 4  

TOP 2 SCORE (%)*

Reuse of material  n=102

53.9

Timely coordination of visits  n=77

49.4

*Excludes don’t know / not applicable / no answer responses

PC: KPI 4

 KPI 4

PC KPI 5: TEQSA is open, transparent and consistent in its dealings with your organisation

TEQSA’s item quality of information under KPI 5 was rated fairly positively, with a top 2 score of 74% (-5% since 2017). The three other items all rated between 54% and 60% - slightly lower than the 2017 results.

Note that the item Consistency of decisions received a relatively high proportion of not applicable ratings.

PC: KPI 5

TOP 2 SCORES (%)

Consistency of information  n=146

58.6

Quality of information on National Register  n=149

74.0

Availability of information  n=152

59.7

Consistency of decisions  n=117

53.8

*Excludes don’t know / not applicable / no answer responses

PC: KPI 5

 KPI 5

PC KPI 6 - TEQSA continues to improve its regulatory framework in consultation with your organisation

Variety of media was the highest scoring item for KPI 6 with a top 2 score of 66%. Note that positive ratings for Making process improvements dropped notably this year: to 52% from 62% in 2017.

PC: KPI 6

TOP 2 SCORE (%)*

Variety of media  n=145

65.5

Engagement  n=154

53.9

Making process improvements  n=138

52.2

*Excludes don’t know / not applicable / no answer responses

PC: KPI 6

 KPI 6

PC: Interactions with TEQSA

Providers were asked to indicate the types of interactions they had with TEQSA in the last 12 months. The most common interactions were Use of TEQSA’s website (94%) and the Use of TEQSA’s guidance / support materials (89%). 

PC: Types of interactions with TEQSA in last 12 months

Bar chart showing types of interactions with TEQSA in last 12 months

PC: Applications

This section outlines providers’ views of the interactions they had with TEQSA while making registration and accreditation applications, including for CRICOS and self-accrediting authority (SAA). The table and chart results within this section are based on a sub-set of the PC survey sample (n=102), that is, only those who indicated that they had a particular interaction in the previous 12 months. 

The following series of tables and charts display the top 2 scores and frequency distributions of answers about aspects of these interactions.

PC: Registration, accreditation and SAA application processes

All items for TEQSA’s Application Process rated between 60% and 70%. The item clarity of the online form and application guide remained key strengths, rated positively by 70% and 69% of respondents.

PC: REGISTRATION, ACCREDITATION AND SAA APPLICATION PROCESSES  

TOP 2 SCORE (%)*

Clarity of the application guide  n=99

69.7

Clarity of the online form  n=98

69.4

Helpfulness of information about how to prepare an application  n=99

65.7

Helpfulness of portal information  n=99

65.7

Clarity of the assessment scope and evidence requirements  n=99

64.6

Any follow up assistance that was required  n=89

61.8

*Includes only respondents who were involved with aspects of registration, accreditation or SAA.
Top 2 scores and n counts exclude don’t know / not applicable / no answer responses.

PC: TEQSA Application process

 TEQSA application process

PC: CRICOS application process

The providers who indicated that they had interacted with TEQSA about CRICOS registration or any other aspect of a CRICOS application in the last 12 months were asked about aspects of their interactions. The table and chart results within this section are based on a sub-set of the PC survey sample (n=74), that is, only those who indicated that they had this particular interaction.

Three of the six items relating to this process had a significantly lower top 2 score this year: Clarity of the application guide (now 68% and 85% in 2017), helpfulness of information about how to prepare an application (now 64% and 79% in 2017) and clarity of the online form (now 60% and 90% in 2017).

PC: CRICOS APPLICATION PROCESS  

TOP 2 SCORE (%)*

Helpfulness of portal information  n=67

71.6

Clarity of the application guide  n=65

67.7

Clarity of the assessment scope and evidence requirements  n=66

66.7

Any follow up assistance that was required  n=63

65.1

Helpfulness of information about how to prepare an application  n=66

63.6

Clarity of the online form  n=67

59.7

*Includes only respondents who were involved with CRICOS registration or other CRICOS applications.
Top 2 scores and n counts exclude don’t know / not applicable / no answer responses.

PC: CRICOS Application process

 CRICOS application process

PC: TEQSA's case management approach 

The vast majority of respondents (86%) indicated TEQSA’s case management approach was very important to their organisation and 10% indicated that it was somewhat important.

PC: Importance of TEQSA's case management approach

Bar chart showing Importance of TEQSA's case management approach

Respondents who had had interaction with a case manager or were involved in an application process were then asked about their experiences of TEQSA’s case management over the last 12 months. Seventeen percent indicated their experiences had improved, while for 53% case management experiences had stayed the same. However, 23% of respondents indicated that their experiences had worsened. Refer to the chart below.

PC: Experiences of TEQSA's case management

Bar chart showing Experiences of TEQSA's case management

Those providers who indicated that case management had improved or worsened were asked to comment about what had happened with their case management over the previous 12 months. 

Positive comments focussed on communicative, consultative, responsive and/or proactive case management.

Negative comments focussed on not having a case manager or not knowing who their case manager was, having multiple case managers in a short time, unresponsiveness of case managers and poor quality case management.

PC: TEQSA's case management approach

Those providers who had interacted with a TEQSA case manager in the last 12 months were asked about different aspects of TEQSA’s case management approach. Ratings for the three items remained relatively consistent with last year’s scores. Responsiveness scored highest at 60%.

PC: CASE MANAGEMENT APPROACH

TOP 2 SCORE (%)*

Responsiveness  n=146

59.6

Consideration of your needs  n=142

51.4

Knowledge of your organisation  n=145

48.3

*Includes only respondents who interacted with case managers in the last 12 months.
Top 2 scores and n counts exclude don’t know / not applicable / no answer responses.

PC: Case management approach

Bar chart showing Case management approach

PC: TEQSA conference 

Ratings for the TEQSA conference were amongst the highest in the PC survey. Of the providers who attended the conference, 91% rated the item opportunity to interact with other delegates as excellent or good. All items scored above 80% in this topic, making it one of the higher scoring topics in the PC survey.

PC: TEQSA CONFERENCE

TOP 2 SCORE (%)*

Opportunity to interact with other delegates  n=97

90.7

The quality of speakers  n=97

85.6

The program  n=97

83.5

Relevance of material presented  n=97

82.5

*Includes only respondents who attended the TEQSA Conference.
Top 2 scores and n counts excludes don’t know / not applicable / no answer responses.

PC: TEQSA Conference

 TEQSA Conference

PC: Guidance and support materials

TEQSA’s guidance and support materials were rated highly, with all top 2 scores rating between 83% and 91%. The relevance of the information and the quality of the information were the highest scoring items (91% and 89%) making this topic one of the highest scoring in the survey, along with the TEQSA conference.

PC: GUIDANCE AND SUPPORT MATERIALS

TOP 2 SCORE (%)*

Relevance of information  n=138

90.6

The quality of the information  n=138

89.1

Usefulness of information  n=138

86.2

Ease of access to that information  n=138

87.7

Amount of information  n=137

83.2

*Includes only respondents who used guidance and support materials.
Top 2 score and n counts exclude don’t know / not applicable / no answer responses.

PC: TEQSA guidance and support materials

 TEQSA guidance and support materials

PC: TEQSA overall

All providers were asked to rate TEQSA as a regulator. Seventy-one percent of respondents who answered this question rated TEQSA’s performance over the last 12 months as either good or excellent - down 9% since 2017.

Note that the total of the excellent and good proportions in the chart below is slightly less than 80%. This is because the chart includes a proportion of don’t know / no answer respondents who have been excluded from the top 2 calculation.

PC: TEQSA as a regulator

 TEQSA as a regulator

VC / CEO comments

To ensure that TEQSA could still give VC / CEOs the opportunity to make personal comments about TEQSA’s performance in the last 12 months, a separate section has been included in the PC survey. All participants were aware of this section / opportunity before answering.

The common themes from the 52 valid suggestions offered within the VC / CEO question were:

  • Positive comments around the value of TEQSA, its activities and approach
  • Negative / adversarial approach, particularly with private and potential providers 
  • Be more of a partner
  • Protracted process, lack of response
  • More interaction / face-to-face communication
  • Little consideration of context.

Again, so as not to lose the value of individual and detailed comments, ASR strongly recommends that relevant TEQSA staff read the (de-identified) verbatim comments from this survey.

Principal Contact: Year comparison

In the last 12 months, principal contacts’ perceptions of TEQSA’s performance declined in many areas; six items decreased significantly—four of these related to the CRICOS application process which we note just over 40% PC survey respondents were involved with in 2017.

Significant or notable declines that affected most responding providers included completeness of information, streamlining to reduce burden, making process improvements, consistency of information and overall performance as a regulator.

Attribute analysis gives a different view of respondent sentiment where some sub-groups had similar but not identical views to those expressed in 2017. However, scores declined across nearly all items, irrespective of sub-group. The most positive sub-groups in 2017 were less positive in 2018.

All scores were lower in 2018 compared with 2016 (where applicable). However a few scores were only slightly lower and well within the survey’s margin of error.

IMPORTANT NOTE: The percentage change between years must be read with the survey’s confidence interval (±4.6%) in mind. This plus / minus percentage is the margin of error in this survey, this year. All measuring processes, irrespective of how the measuring is conducted, have a margin of error. It means that any change within this margin could be considered a measuring error. If exactly the same survey was run again with the same people, scores could realistically change within this interval and still be considered an accurate or true result.

Note that some differences in ratings between years for some of the items have not been highlighted in bold because these were not statistically significant at the 95% confidence level. This apparent discrepancy is about the way in which significant difference is calculated and the test’s reliance on the overall spread or variance of answers to a question.

Only significant differences between the 2018 and 2017 responses have been highlighted below.

Statistically significantly different (p<0.05) results between 2018 and 2017 highlighted in bold

ITEM

Count 2018

2018

% +VE

2017

% +VE

2016

% +VE

± 2017 / 2018 only

%

CRICOS Application process: Clarity of the online form

67

59.7

89.9

84.1

-30.2

CRICOS Application process: Clarity of the application guide

65

67.7

84.5

76.1

-16.8

KPI 2: Completeness of information

123

55.3

71.2

81.0

-15.9

CRICOS Application process: Helpfulness of information about how to prepare an application

66

63.6

79.2

70.4

-15.5

CRICOS Application process: Any follow up assistance that was required

63

65.1

79.1

84.8

-14.0

KPI 3: Actions proportionate to risks

116

47.4

60.9

65.0

-13.5

CRICOS Application process: Helpfulness of portal information

67

71.6

84.5

86.8

-12.9

KPI 1: Streamlining to reduce burden

138

47.1

59.2

71.4

-12.1

CRICOS Application process: Clarity of the assessment scope and evidence requirements

66

66.7

77.5

73.2

-10.8

KPI 6: Making process improvements

138

52.2

61.8

69.9

-9.7

KPI 5: Consistency of information

152

58.6

67.6

74.8

-9.1

Overall performance

149

71.1

79.7

82.3

-8.6

KPI 2: Clarity of information

122

60.7

68.9

80.5

-8.3

KPI 1: Opportunity to give feedback

144

64.6

72.8

77.0

-8.2

Application process: Clarity of the online form

98

69.4

77.5

79.3

-8.1

KPI 2: Quality of regulatory information

154

80.5

88.6

80.8

-8.1

KPI 2: Reasonable opportunity to address matters

115

64.3

72.3

80.7

-7.9

KPI 2: Communicating streamlining initiatives

144

51.4

59.2

78.9

-7.8

Application process: Helpfulness of portal information

99

65.7

72.8

73.0

-7.2

Application process: Any follow up assistance that was required

89

61.8

68.5

80.0

-6.7

KPI 5: Consistency of decisions

117

53.8

60.6

73.7

-6.7

Application process: Clarity of the application guide

99

69.7

76.0

76.9

-6.3

Application process: Clarity of the assessment scope and evidence requirements

99

64.6

70.8

71.1

-6.2

Conference: The program

97

83.5

89.2

N/A

-5.7

KPI 5: Quality of information on National Register

146

74.0

79.1

80.5

-5.1

Guidance & Support materials: The quality of the information

138

89.1

94.1

N/A

-5.0

KPI 6: Engagement

154

53.9

58.8

78.9

-4.9

Conference: The quality of speakers

97

85.6

90.4

N/A

-4.8

KPI 2: Relevance of information

154

82.5

87.1

86.8

-4.7

Guidance & Support materials: Usefulness of information

138

86.2

90.6

N/A

-4.4

Conference: Relevance of material presented

97

82.5

86.7

N/A

-4.3

KPI 4: Timely coordination of visits

77

49.4

53.3

75.0

-4.0

Case mgt approach: Consideration of your needs

142

51.4

55.4

70.9

-4.0

Case mgt approach: Knowledge of your organisation

145

48.3

52.2

71.1

-3.9

KPI 5: Availability of information

149

59.7

63.3

61.2

-3.6

Case mgt approach: Responsiveness

146

59.6

62.9

78.3

-3.3

Application process: Helpfulness of information about how to prepare an application

99

65.7

68.8

66.3

-3.1

KPI 4: Reuse of material

102

53.9

56.4

70.8

-2.5

Guidance & Support materials: Relevance of information

138

90.6

92.9

N/A

-2.4

KPI 2: Timeliness of information after making a decision

113

52.2

54.5

76.5

-2.2

Conference: Opportunity to interact with other delegates

97

90.7

91.7

N/A

-0.9

Guidance & Support materials: Amount of information

137

83.2

81.2

N/A

2.0

KPI 3: Consultative approach

132

56.1

53.2

56.6

2.9

KPI 6: Variety of media

145

65.5

61.8

68.8

3.7

Guidance & Support materials: Ease of access to that information

138

87.7

82.4

N/A

5.3

*Top 2 scores and n counts exclude don’t know / not applicable / no answer responses.
N/A – question was not asked in 2016

Provider attribute analysis

Attribute analysis is conducted to identify where there are similarities and differences between providers. It provides insight that overall or aggregated analysis cannot. It helps answer the questions “Do some groups perceive that they are treated differently?” and “Do sub-groups have similar or different views?” Essentially this analysis is used to discover whether or not TEQSA has the same or different interactions with various sub-groups and conversely if these sub-groups have the same or different perceptions of TEQSA. 

The following section presents the results of principal contact responses using nine provider attributes: 

  1. Self-accrediting authority
  2. 2017 risk to financial position
  3. 2017 risk to students
  4. Category
  5. Provider size
  6. State
  7. Registered Training Organisation (RTO) activity
  8. CRICOS registration and 
  9. Market groupings. 

The table below and which continues on the following page shows the sub-sets used in the analysis of each attribute.

ATTRIBUTE

SUB-GROUPS

n

% of RESPONSE SAMPLE

Self-accrediting authority+

No

81

63.8

 

Yes / Part yes

46

36.2

2017 Risk to financial position

High / moderate

37

28.9

 

Low

74

57.8

 

Other

17

13.3

2017 Risk to students

High / Moderate

59

46.5

 

Low

55

43.3

 

Other

13

10.2

Category

University*

37

23.9

 

Higher Education Provider (HEP)

90

58.1

 

Prospective HEP

28

18.1

Provider size

<100

34

29.3

 

100 - 499

21

18.1

 

500 - 4,999

28

24.1

 

5,000 - 19,999

17

14.7

 

≥ 20,000

16

13.8

State

NSW

72

46.5

 

VIC

38

24.5

 

QLD

15

9.7

 

WA

12

7.7

 

SA

12

7.7

 

Other

6

3.9

Active RTO

No

61

48.0

 

Yes

66

52.0

CRICOS# registered

No

27

21.3

 

Yes

100

78.7

Market groupings

Faith based

13

8.3

 

Miscellaneous^

27

17.3

 

For profit

36

23.1

 

University

37

23.7

 

Other (not for profit, non-faith based)

14

9.0

 

N/A as prospective HESPs

28

17.9

 

No value available

1

0.6

+ Note that the attribute analysis below excludes the proposed higher education providers (n=28).
*University includes Australian university, Australian university of specialisation and overseas university
^Includes Government Agencies, Pathways, Professional Bodies and TAFEs
#stands for Commonwealth Register of Institutions and Courses for Overseas Students

Only items which were statistically significantly different at the 95% confidence level have been included in the sub-group comparison tables below. These differences are presented using top 2 (% positive) scores only and were analysed using a z test. Statistically significantly higher results are highlighted in bold.

Top 2 scores were calculated using only the number of respondents who chose a rating point answer. In other words don’t know, not applicable and no answers (blank) were excluded from statistical calculations. A top 2 score of 100% means that all respondents who answered a question rated the item as good or excellent.

Important note: there were NO significant differences in any items for the various sub-groups within provider size, state and on the yes/no question relating to concern about sector quality /reputation. Results for these attributes were not included in the report.

PC: Self-accrediting authority

For all items in the table below, providers who had self-accreditation status rated significantly higher than providers who did not have this authority. This should not be a surprise to TEQSA as higher quality providers are likely to have this authority and also have a more positive view of TEQSA as a result of being granted the authority. They also probably have fewer interactions with TEQSA.

Significantly higher results highlighted in bold.

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM

SELF-ACCREDITING AUTHORITY
TOP 2 SCORES (%)

  No
max n=84
Yes / Part yes
max n=46

KPI 1: Opportunity to give feedback

60.0

81.0

KPI 2: Communicating streamlining initiatives

48.1

67.5

KPI 2: Reasonable opportunity to address matters

60.3

83.9

KPI 2: Timeliness of information after making a decision

46.2

67.6

KPI 2: Clarity of information

53.6

78.4

KPI 2: Completeness of information

50.0

70.3

KPI 2: Quality of regulatory information

78.8

93.3

KPI 3: Actions proportionate to risks

40.6

65.7

KPI 5: Quality of information on National Register

65.8

83.3

KPI 5: Consistency of information

52.5

71.1

KPI 6: Engagement

45.0

66.7

CRICOS Application process: Any follow up assistance that was required

51.7

79.3

Case mgt approach: Responsiveness

53.9

73.3

Overall performance

61.8

88.6

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.

PC: 2017 Risk to financial position

The providers that TEQSA rated as having a low risk to financial position in 2017 rated TEQSA’s performance highest on all items in the below table.

Significantly higher results highlighted in bold.

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM 

2017 RISK TO FINANCIAL POSITION
TOP 2 SCORES (%)

  High / mod
max n=37
Low
max n=74
Other
max n=17^

KPI 2: Clarity of information

46.9

72.9

56.3

KPI 2: Relevance of information

83.8

88.9

64.7

KPI 3: Actions proportionate to risks

30.3

59.6

53.3

KPI 5: Quality of information on National Register

69.4

80.9

40.0

Guidance & Support materials: The quality of the information

91.9

97.0

73.3

Guidance & Support materials: Relevance of information

91.9

97.0

80.0

Guidance & Support materials: Ease of access to that information

83.8

97.0

93.3

Guidance & Support materials: Usefulness of information

89.2

93.9

73.3

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.
^Some cell numbers (n counts) are very small so results should be interpreted with extreme caution.

PC: 2017 Risk to students

The providers that TEQSA rated as having a low risk to students in 2017 rated TEQSA’s performance highest on all items in the below table. There were a considerable number of differences for this attribute, including for TEQSA’s overall performance.

Note the large differences in TEQSA’s overall performance rating. 

Significantly higher results highlighted in bold.

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM

2017 RISK TO STUDENTS
TOP 2 SCORES (%)

  High / mod
max n=59
Low
max n=55
Other
max n=13^

KPI 1: Streamlining to reduce burden

38.2

62.5

46.2

KPI 1: Opportunity to give feedback

56.9

82.4

53.8

KPI 2: Reasonable opportunity to address matters

58.0

89.2

50.0

KPI 2: Clarity of information

52.0

77.3

58.3

KPI 2: Quality of regulatory information

81.0

92.6

61.5

KPI 2: Relevance of information

84.5

90.7

53.8

KPI 3: Consultative approach

47.4

71.4

54.5

KPI 3: Actions proportionate to risks

30.2

72.5

63.6

KPI 4: Timely coordination of visits

36.4

75.0

37.5

KPI 5: Quality of information on National Register

68.4

83.7

41.7

KPI 6: Engagement

43.1

66.7

46.2

CRICOS Application process: Any follow up assistance that was required

45.8

82.8

60.0

Case mgt approach: Responsiveness

46.4

75.5

66.7

Case mgt approach: Consideration of your needs

39.3

66.0

58.3

Guidance & Support materials: The quality of the information

94.5

96.2

63.6

Guidance & Support materials: Relevance of information

94.5

96.2

72.7

Guidance & Support materials: Usefulness of information

90.9

94.2

63.6

Overall performance

61.8

86.5

61.5

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.
^Some cell numbers (n counts) are very small so results should be interpreted with extreme caution.

PC: Category

Where there were differences, universities provided significantly higher ratings compared with HEP and prospective HEP groups, as displayed in the table below. Ratings for TEQSA overall as a regulator were consistent across the groups.

Significantly higher results highlighted in bold.

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM

CATEGORY - TOP 2 SCORES (%)

  University^
max n=37
HEP
max n=90
Prospective HEP
max n=28

Guidance & Support materials: The quality of the information

97.0

90.5

70.0

Guidance & Support materials: Ease of access to that information

90.9

92.9

60.0

Guidance & Support materials: Usefulness of information

100.0

85.7

65.0

KPI 2: Communicating streamlining initiatives

67.7

50.0

33.3

KPI 2: Quality of regulatory information

91.7

80.9

64.3

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.
^ includes Australian university, Australian university of specialisation and overseas university
HEP: Higher Education Provider

PC: Registered Training Organisation (RTO) activity

Non-active RTOs rated all items in the table below higher than RTOs that were active. Ratings for TEQSA overall as a regulator were consistent across the groups.

Significantly higher results highlighted in bold.

 

RTO ACTIVIY - TOP 2 SCORES (%)

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM

Active RTO
max n=66

Non-active RTO
max n=61

KPI 1: Streamlining to reduce burden

37.3

59.6

KPI 4: Reuse of material

40.8

67.4

KPI 6: Making process improvements

41.4

64.9

CRICOS Application process: Helpfulness of information about how to prepare an application

46.9

82.8

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.

PC: CRICOS registration

Only two items significantly differed between CRICOS-registered organisations and organisations that were not CRICOS registered. Refer to the table below.

Significantly higher results highlighted in bold.

 

CRICOS - TOP 2 SCORES (%)

SIGNIFICANTLY DIFFERENT ITEMS 
TOPIC / ITEM
CRICOS 
registered 
max n=100
Not CRICOS registered
max n=27

KPI 1: Opportunity to give feedback

71.9

50.0

KPI 5: Availability of information

54.6

80.8

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.

PC: Market groupings

Universities scored significantly higher than nearly all other market groups on most of the items in the table below. There were considerable differences between sub-groups for this attribute, but TEQSA’s overall performance rating was NOT an area of difference. Where there were significant differences, the For profit group was generally lowest scoring. This overall view was reinforced in open-ended comments with comments like TEQSA is adversarial, suspicious and not understanding of their context. 

Significantly higher results highlighted in bold.

SIGNIFICANTLY DIFFERENT ITEMS
TOPIC / ITEM

MARKET GROUPINGS^ - TOP 2 SCORES (%)

  Faith based
n=13^
Miscellaneous n=27^ For profit
n=36
University
n=37

Other
(not for profit, non-faith based)
n=14^

N/A
as prospective HEP
n=28^

KPI 2: Clarity of information

72.7

73.9

35.3

71.4

90.0

46.7

KPI 2: Quality of regulatory information

92.3

96.3

61.1

91.7

92.3

64.3

KPI 2: Timeliness of information after making a decision

70.0

66.7

27.3

68.0

60.0

46.2

KPI 3: Actions proportionate to risks

70.0

62.5

15.2

65.4

63.6

27.3

KPI 5: Consistency of decisions

63.6

65.2

33.3

57.7

90.0

38.5

Guidance & Support materials: The quality of the information

100.0

91.7

85.3

97.0

92.3

70.0

Overall performance

91.7

72.0

44.1

85.7

85.7

67.9

Case mgt approach: Consideration of your needs

83.3

58.3

33.3

52.9

66.7

43.5

Total n varies by item because not all respondents were eligible to answer all questions. The maximum possible n is displayed in the table header.
^Some cell numbers (n counts) are very small so results should be interpreted with extreme caution.
#Includes Government Agencies, Pathways, Professional Bodies and TAFEs
+Includes not for profit and non-faith based

Peak/professional/student body survey key findings

This section outlines the key findings from TEQSA’s peak, professional and student body survey. For all items, top 2 scores are presented along with a frequency distribution. Results are presented by topic, in the same order as presented to respondents in the online questionnaire. The most common themes within free text comments follow.

Note: Due to the relatively small number of respondents for this section (a total of 24 peak / professional/student bodies answered the survey) these results should be treated with considerable caution and only indicative at best.

In this section, the term peak, professional and student body has been abbreviated to PPSB.

PPSB: Interaction with TEQSA

The chart below displays the types of direct or indirect interactions PPSBs had with TEQSA in 2017/2018. The most common type of interaction with TEQSA was direct first-hand experience (88%). Fifty percent of PPSBs had indirectly dealt with TEQSA or had feedback from members.

PPSB: Interactions with TEQSA in last 12 months 

 Interactions with TEQSA in last 12 months

PPSB: Scores for all questions

From a PPSB perspective, TEQSA performed well on all KPIs. Refer to the table and chart below. The KPIs around impact, risk approach and approach were key strengths for TEQSA from a PPSB perspective. TEQSA’s performance on the six KPIs varied from a high of 94% (KPI 1) to a low of 79% (KPI 6). Note that for KPIs 1, 3 and 4 there was a considerable proportion of don’t know / no answer responses.

PPSB: KPIs 

 

n

TOP 2 SCORE (%)

KPI 1  Impact

Regulation by TEQSA does not unnecessarily impede the efficient operation of your organisation

16

93.7

KPI 2  Communication

TEQSA’s communication with your organisation is clear, targeted and effective

23

87.1

KPI 3  Risk approach

Regulatory actions undertaken by TEQSA for your organisation are proportionate to the risks being managed

15

93.3

KPI 4  Compliance / monitoring

TEQSA has a streamlined and co-ordinated approach to compliance and monitoring for your organisation

15

80.0

KPI 5  Approach

TEQSA is open, transparent and consistent in its dealings with your organisation

23

91.3

KPI 6  Continuous improvement

TEQSA continues to improve its regulatory framework in consultation with your organisation

19

79.0

Overall

TEQSA performance over the last 12 months as a regulator

24

100.0

PPSB: TEQSA ratings

 TEQSA ratings

PPSB: Year comparison

Between 2017 and 2018 ratings for all but two of the seven items in the below table increased, with the biggest increase in top 2 score for KPI 6, up 12%. 

Due to very small sample sizes these results should be treated as indicative only. The sample sizes are too small to conduct reliable or valid comparative statistical analysis.

ITEM

 

2018
n

2018
TOP 2 SCORE (%)

2017
TOP 2 SCORE (%)

± %

KPI 6: Continuous improvement

TEQSA continues to improve its regulatory framework in consultation with your organisation

19

78.9

66.7

12.3

KPI 3: Risk approach

Regulatory actions undertaken by TEQSA for your organisation are proportionate to the risks being managed

15

93.3

89.5

3.8

KPI 5: Approach

TEQSA is open, transparent and consistent in its dealings with your organisation

23

91.3

88.9

2.4

KPI 2: Communication

TEQSA’s communication with your organisation is clear, targeted and effective

23

87.0

84.6

2.4

KPI 1: Impact

Regulation by TEQSA does not unnecessarily impede the efficient operation of your organisation

16

93.8

91.7

2.0

Overall

TEQSA performance over the last 12 months as a regulator

24

85.7

87.5

-1.8

KPI 4: Compliance / monitoring

TEQSA has a streamlined and co-ordinated approach to compliance and monitoring for your organisation

15

80.0

85.0

-5.0

Survey comparisons

For this piece of analysis, a simple average of the top 2 scores for each item within a topic (KPI) of the principal contact survey was calculated. These six average top 2 scores were then compared with the top 2 scores of the same KPIs within the PPSB survey, along with the top 2 scores of TEQSA’s overall rating as a regulator (the same overall regulator performance question was asked in both surveys). The comparative results are displayed in the chart below.

It shows that PPSBs were more positive about TEQSA’s performance on all KPIs; all had a top 2 score of 79% or higher for PPSBs, compared to top 2 scores ranging between 52% and 71% for principal contacts.

KPI and overall comparison across 2 target groups

Bar chart showing KPI and overall comparison across 2 target groups

Conclusions

After an excellent result in TEQSA’s first stakeholder survey (2016), this third year of surveying has a lower percentage of respondents rating TEQSA excellent or good on all indicators and this decline is universal across sub-groups within the provider population. There are some providers (usually low risk and self-accrediting) who are extremely happy with TEQSA’s performance while for profit providers are less positive. 

This year’s results should continue to give TEQSA clear guidelines on where to focus service initiatives. Many of the suggestions for improvements or change are similar to previous surveys. The majority of providers continue to rate TEQSA excellent or good in each of its 6 KPIs and the relationship between TEQSA and the PPBS results continue to be extremely positive. Providers have not notably changed their requests for areas of change.

The activities or processes that providers think TEQSA is doing well include:

  • Provision of guidance and support materials
  • TEQSA’s annual conference
  • The quality and relevant of regulatory information
  • Its overall performance as a regulator

Areas that providers believe TEQSA needs to improve include:

  • CRICOS applications
  • Streamlining regulatory burden
  • Speed of response when making decisions
  • Consultation and engagement
  • Case management, in particular, case managers’ knowledge of providers and consideration of their needs as well as more visits / face-to-face meetings.

Peak, professional and student bodies have much more positive views about TEQSA’s performance than providers.

Importantly, overall TEQSA was fairly well-regarded by providers and extremely well-regarded by peak bodies as a regulator assuring the quality of Australia’s higher education.

Appendix A: Table of item abbreviations

TOPIC

ITEM FULL NAME

ABBREVIATED NAME

KPI 1

Streamlining its regulatory processes and practices to reduce (or positively affect) administrative burden for your organisation

Streamlining to reduce burden

 

Providing your organisation with the opportunity to give feedback on proposed changes to TEQSA’s practices (including streamlining initiatives)

Opportunity to give feedback

KPI 2

Communicating its streamlining initiatives to your organisation

Communicating streamlining initiatives

 

Providing a reasonable opportunity to address matters relevant to a regulatory decision, prior to a final decision being made

Reasonable opportunity to address matters

 

Timeliness of information provided by TEQSA after TEQSA makes a regulatory decision

Timeliness of information after making a decision

 

Clarity of information about TEQSA's regulatory decisions

Clarity of information

 

Completeness of information about TEQSA's regulatory decision

Completeness of information

 

Quality of information on TEQSA's regulatory policies and processes  provided through TEQSA’s website and newsletters

Quality of regulatory information

 

Relevance of information  on TEQSA’s regulatory policies and processes  provided through TEQSA’s website  and newsletters

Relevance of information

KPI 3

The consultative approach taken to confirm the annual risk assessment results with your organisation

Consultative approach

 

Regulatory actions undertaken by TEQSA for your organisation are proportionate to the risks being managed

Actions proportionate to risks

KPI 4

TEQSA’s reuse of material provided by your organisation for a range of regulatory matters

Reuse of material

 

Timely coordination of TEQSA staff visits to your organisation

Timely coordination of visits

KPI 5

Availability of information on trends and observations on sector performance

Availability of information

 

Quality of information provided on the National Register (showing the results of the regulatory decisions)

Quality of information on National Register

 

Consistency of information provided to your organisation

Consistency of information

 

Consistency of TEQSA's decisions about your organisation

Consistency of decisions

KPI 6

Using a variety of media and channels to communicate sector-wide updates

Variety of media

 

Direct engagement with your organisation through briefings and roundtables

Engagement

 

Making improvements to its processes and policies in areas that impact your organisation

Making process improvements

Application process

Clarity of the application guide (easy to understand)

Clarity of the application guide

 

Clarity of the assessment scope and evidence requirements

Clarity of the assessment scope and evidence requirements

 

Helpfulness of information about how to prepare an application

Helpfulness of information about how to prepare an application

 

Helpfulness of information on how to use the provider portal (for preparing and submitting applications online)

Helpfulness of portal information

 

Clarity of the online form

Clarity of the online form

 

Any follow up assistance that was required

Any follow up assistance that was required

CRICOS application process

Clarity of the application guide (easy to understand)

Clarity of the application guide

 

Clarity of the assessment scope and evidence requirements

Clarity of the assessment scope and evidence requirements

 

Helpfulness of information about how to prepare an application

Helpfulness of information about how to prepare an application

 

Helpfulness of information on how to use the provider portal (for preparing and submitting applications online)

Helpfulness of portal information

 

Clarity of the online form

Clarity of the online form

 

Any follow up assistance that was required

Any follow up assistance that was required

Case mgt approach

Responsiveness to the needs of your organisation

Responsiveness

 

Knowledge of your organisation’s specific needs / issues / environment

Knowledge of your organisation

 

Consideration of your organisation’s specific needs / issues / environment for tailoring the application process

Consideration of your needs

Conference

The program

The program

 

The quality of speakers

The quality of speakers

 

Relevance of material presented

Relevance of material presented

 

Opportunity to interact with other delegates

Opportunity to interact with other delegates

Guidance and support materials

The quality of the information

The quality of the information

 

Relevance of information

Relevance of information

 

Ease of access to that information

Ease of access to that information

 

Usefulness of information

Usefulness of information

 

Amount of information

Amount of information

Overall

Overall: TEQSA's performance over the last 12 months as the regulator assuring the quality of Australian higher education

Overall performance

Appendix B: Questionnaires used in 2018 surveys

The appendix contains the full questionnaires used in the principal contact survey and the peak, professional and student body survey.