THE 44TH INTERNATIONAL ASSOCIATION FOR
EDUCATIONAL ASSESSMENT (IAEA) ANNUAL CONFERENCE

Pre Conference Workshops

Programme For Pre-Conference Workshops

 

 

 

Sunday, September  9

14.00 – 15.30

Workshop A 

What can we learn from Progress in International Reading Literacy Study (PIRLS)? Use of international assessment data in educational research 

Dr Grace Grima,  Pearson UK

Dr  Joshua McGrane, Oxford University Centre for Educational Assessment

Dr Therese N. Hopfenbeck, Oxford University Centre for Educational Assessment

Why you should attend this workshop:

If you are interested in more detailed information on international large-scale studies, this workshop will give you an overview of possibilities for both research and practice through the case of PIRLS 2016. If you have been directly involved in large national and international assessment research projects, we would value your input as well. 

You will have the opportunity to discuss what potentials and limitations International Large Scale Assessment (ILSA) studies can offer, and share from your own experiences with other delegates. 

Who this Workshop is for:

You do not need to have any prior knowledge of international large-scale studies to participate in the conversations. We will present our experience from running the PIRLS 2016 study in England, and how we have worked on engaging practitioners such as teachers through our PIRLS for Teachers study in Oxford. If you are interested in the use of ILSA data, either as a researcher, test developer or policy maker, this workshop can offer you food for thought.

Overview:

The workshop will give an overview of the PIRLS study in general, and experiences and results from the PIRLS 2016 study in England in particular. We will share released PIRLS items and discuss how we have facilitated discussions with teachers in England on what we can learn from PIRLS to support pupils reading competence. We would be interested in bringing in experiences from other countries from participants at IAEA 2018. We will provide materials, so there is no need to bring anything.

About the Workshop Leaders:

 

Dr Grace Grima is Associate Professor and Director of Research at Pearson UK.

Grace completed her Ph.D in 1999 on the national educational monitoring project (NEMP) in New Zealand where she was Commonwealth student at the University of Otago, In her previous role as Director General for Quality and Standards at the Ministry of Education and Employment in Malta (2007- 2013), Grace was national research co-ordinator for PIRLS 2011 and was involved in other international educational assessment research projects. In her current role within Pearson, Grace is the national research co-ordinator for PIRLS 2016 and also for TIMSS 2019. Grace is also an Honorary Senior Research Associate, Department of Curriculum, Pedagogy & AssessmentUCL Institute of Education, London, UK (2015 – to date).

 

Dr. Joshua McGrane is Deputy Director of the Oxford University Centre for Educational Assessment. He completed his university medal-winning PhD in Quantitative Psychology and BA (Hons I) in Psychology at the University of Sydney.

He has been a Postdoctoral Fellow in the Graduate School of Education at the University of Western Australia and also worked as a Psychometrician for the Centre for Education Statistics and Evaluation (CESE) in the New South Wales Department of Education. He was the lead author of the PIRLS 2016 National Report for England. He also has extensive experience across academic and government contexts in educational assessment, including psychometric analyses and innovation of state and national-level assessments. 

Dr Therese N. Hopfenbeck is Associate Professor and Director of the Oxford University Centre for Educational Assessment. Her research interests include large-scale comparative assessments, different models of classroom assessment and self-regulation. Therese was the Research Manager of PIRLS 2016 in England and Principal Investigator for the Knowledge Exchange Program PIRLS for Teachers, funded by ESRC and is currently Principle Investigator for Assessment for Learning in Africa (AFLA): Improving Pedagogy and Assessment for Numeracy in Foundation Years funded by ESRC-DfID. She is Lead Editor of the international research journal Assessment in Education: Principles, policy and practice and member of the PISA 2021 Questionnaire Expert Group.

Room Arumugam Block – AB 1.1

14.00 – 15.30


Workshop B  
Construct Validation of a Scientific Reasoning Test using Rasch Measurement Theory

David Andrich,  University of Western Australia, Australia.

Diana Ng,  Oxford University Centre for Educational Assessment, UK.

Why you should attend this workshop:

Participants will learn some fundamental psychometric concepts and understand broadly how these concepts are used to study the validity of an educational assessment. Drawing on a test of scientific reasoning as an illustration, participants will examine the construct basis of this cognitive proficiency in some well-known classroom science tests (e.g., Lawson’s Test) and large-scale international assessments (e.g., PISA, TIMSS). As a comparison, participants will be introduced to a new test constructed from a novel conceptualisationof scientific reasoning. By analysing the performance of some items from the test, they will appreciate the roles psychometric data and qualitative understanding of the construct play in shaping decisions about item and test quality. In particular, it will be emphasized that various tests of fit should be used jointly and that any misfit should be referenced to the construct and other items of the test and not used mechanistically to delete or modify items. This appreciation will help participants identify ways of enhancing current tests of scientific reasoning to impact teaching and learning. 

 

Who this Workshop is for:

Science educators, item writers and testing specialists. Participants do not require prior knowledge to attend. 

Overview:

The workshop first discusses the foundations of Rasch analysis and the measurement advantages confer by this psychometric model. In preparation for applying this understanding to the educational context, the workshop next introduces the measurement of scientific reasoning. Through group discussions and exercises, participants analyse selected items from current science tests to understand various approaches for assessing higher-order thinking skills. The second half of the workshop introduces the design parameters of a new scientific reasoning test. 430 12-year old primary pupils in Singapore took the test, andthe Rasch outputs of their performance in some polytomous and dichotomous items are used for follow-up discussions.  These discussions centre on the use of psychometric information and design parameters to inform item and test quality. 

About the Workshop Leaders:

David Andrich BSc, BEd (Hons.), MEd (UWA), PhD (Chic) FASSA graduated in Mathematics and Education and received a Fulbright Award for study in the Measurement, Evaluation and Statistical Analysis Program at The University of Chicago where his dissertation earned the Susan Colver Rosenberger prize for the best research thesis in the Division of the Social Sciences in 1973. He returned to his appointment at The University of Western Australia, and in 1985 was appointed Professor of Education at Murdoch University in Western Australia where he held the position of Dean for two periods. In 2007 he returned to The University of Western Australia as the Chapple Professor of Education. He is also Visiting Professor in Educational Assessment at The University of Oxford. He has also been a visiting Professor for three periods at The University of Trento in Italy and visiting Scholar at the University of Copenhagen and The University of Chicago. He has held major research grants from the Australian Research Council continuously since 1985. He has also conducted commissioned research at both the national and state levels, and has been a member of expert advisory committees at national and state levels concerned with large-scale assessments. In 1990, he was elected Fellow of the Academy of Social Sciences of Australia for his contributions to measurement in the social sciences. He is especially known for his work in modern test theory, and in particular Rasch measurement theory, where his work has ranged from the philosophy of measurement, through model exposition and interpretation, to software development. He has given invited addresses at many conferences and has published in Educational, Psychological, Sociological and Statistical journals. He is the author of Rasch Models for Measurement (Sage). 

Diana Ng Yee Ping BSc, PGDE (Dist.), MEd (NTU). She is a fourth-year doctorate student at the Oxford University Centre for Educational Assessment. She was a former teacher and examiner of national examinations in Singapore. She received numerous awards during her teaching career including the country’s highest honour for outstanding educators in 2008 – the President’s Award for Teachers. In 2012, she received her Master of Education (Educational and Psychological Measurement and Evaluation) degree from Nanyang Technological University in Singapore. For the degree, she received the certificate of commendation for the 2012 Singapore Teacher’s Union Gold Medal as one of the top three graduates of the course. Her doctoral thesis examined the construct validity of a scientific reasoning test for primary school children in Singapore. The thesis addresses the need to develop valid and reliable instruments for testing reasoning in science learning. Her professional profile is listed here: http://oucea.education.ox.ac.uk/students/diana-ng-yee-ping/

Room Arumugam Block – AB 1.2

14.00 – 15.30


Workshop C 
Achieving in Content Through Language: Assessing Bilingual and Multilingual Learners

Stuart Shaw, Cambridge Assessment, U

Why you should attend this workshop:

This workshop is grounded in practicalities and offers substantial guidance to assessment researchers and practitioners who are currently implementing (or thinking of implementing) bilingual education programmes. The focus is on learning and assessment through a second language (L2), often referred to as an additional language, as well as approaches to support the first language (L1). In this workshop, English constitutes the L2.

Who this Workshop is for:

English is a global language – a lingua franca or lingua mondo. In education, it is a major medium of instruction and assessment for the programmes of international awarding bodies. Such programmes are increasingly offered in a variety of educational and bilingual contexts, and for many international students English is an additional language – their second language, or perhaps their third language. Bilingual assessment research has, therefore, important implications for IAEA participants. In particular, for bilingual learners (for example, senior staff, content teachers and students) and for assessment agencies (for example, test constructors, question paper setters and examiners). The workshop is envisaged as a resource for postgraduate students in educational measurement and assessment, for those with an interest in teaching, curriculum planning and materials development, for key practitioners in assessment agencies who wish to gain a deeper understanding of bilingual assessment, for those with an academic interest in bilingual assessment, and for the novice who should be able to benefit from attending the workshop.

 

Overview:

The workshop will consist of three short sessions with opportunities for audience interaction. The first session will briefly outline theoretical perspectives relating to bilingualism (and the difficulty in defining such a concept), bilingual educational approaches (of which there are many) and bilingual assessment (considered a ’thorny’ issue).  In the second session, insights from international language awareness research, which have led to guidance for schools with bilingual learners (such as senior staff, content teachers and students) and for assessment agencies (such as test constructors, question-paper setters and examiners), are shared. In particular, the workshop will show how research outcomes have assisted in raising L2 awareness in all stages of the development of question papers, mark schemes and examiners’ reports. (The findings have already contributed to the question-writing process: question setters need to be aware of potential language issues confronting an international candidature.) The link between English International assessments and The Common European Framework of Reference for Languages (CEFR) is also explored. Estimates of minimum CEFR language levels in order to aid teachers in the preparation of their students for Cambridge International programmes of learning and summative assessment will be provided. There will also be an opportunity to discuss how the research outcomes have informed the construction of the beginnings of an academic language scale

In a brief concluding session, the significance of the outcomes described – which not only align with previously reported findings in the professional literature but offer new insights, will be discussed.

 

Preparation for the workshop:

Useful (though not compulsory) reading: 

·      Shaw, S. D., & Imam, H. C. (2013). Assessment of international students through the medium of English: ensuring validity and fairness in content-based examinations. Language Assessment Quarterly, 10, 452–475

·      Shaw, S. D., Imam, H., C. & Hughes, K. (2015). Language Rich: insights from multilingual schools. Cambridge: CUP (Chapter 6)

 

About the Workshop Leader:

Stuart Shaw has worked for Cambridge Assessment since January 2001 where he is particularly interested in demonstrating how Cambridge Assessment seeks to meet the demands of validity in its assessments. Stuart has a wide range of publications in English second language assessment and educational research journals. Recent books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008), Validity in Educational and Psychological Assessment (Newton & Shaw, 2014), and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). He is currently working with Isabel Nisbet on a book provisionally entitled: Is assessment fair?


Room Arumugam Block AB 2.1 

To register for the Pre Conference Workshops, please email 
Fionam@fm-int.com, as there are limited spaces. 

 

 

Terms & Conditions
Your use of this website is subject to this Disclaimer and Copyright Statement. By using this website you agree to the terms and conditions and Copyright Statement. If you do not agree with them please do not use this website. The information contained in this website is provided in good faith but no warranty, representation, statement or undertaking is given either regarding such information or regarding any information in (or any software at) any other website connected with this website through any hypertext or other links (including any warranty, representation, statement or undertaking that any information or the use of any such information either in this website or any other website complies with any local or national laws or the requirements of any regulatory or statutory bodies) and any warranty, representation, statement or undertaking whatsoever that may be expressed or implied by statute, custom or otherwise is hereby expressly excluded. The use of this website and any information on this website or any other website (or of any software at any other site) is entirely at the risk of the user. Under no circumstances shall IAEA or any of its subsidiaries be liable for any costs, losses, expenses or damages (whether direct or indirect, consequential, special, economic or financial including any losses of profits) whatsoever that may be incurred through the use of any information contained in this website or in any other website. Nothing contained in this website shall be deemed to be either any advice of a financial nature to act or not to act in any way whatsoever or any invitation to invest or deal in any form of investment including shares, bonds, ADSs or securities. This website may contain inaccurate information. IAEA is under no responsibility to update or correct any such information or to even maintain this website. IAEA reserves its right to change any information or any part of this website without notice.

Copyright Statement
Copyright on all of the images on this web site are owned by IAEA or by others and used under licence, in accordance with the terms of the Copyright Designs and Patents Act 1988.

IAEA (“We”) commit ourselves to protecting and respecting your privacy when visiting our website.

This privacy policy sets out how we will process and use any personal data that you provide to us, or that we may collect as a result of visiting www.iaea2018.org. Please read the following information to understand our practices with regards to your personal data and how we will treat it.

INFORMATION WE MAY COLLECT FROM YOU

We may collect and process the following data about you:
• Any information that you provide by filling in forms on our site. This includes provided information at the time of registering to use our site, posting material, requesting further services or subscribing to any service offered by us on the site. We might also request further information and if you should report a problem with our site.
• We may keep a record of any correspondence if you should contact us.
• Details of the resources you access and of your visits to our site
• Details of transactions you may carry out through our site and of the completion of your orders.

IP ADDRESSES AND COOKIES

We may collect information about your computer; this is statistical data concerning your browsing actions and does not identify you individually. This information includes your IP address where available, operating system and browser type, for system administration and to report collective information to our advertisers. We may also collect information about your general internet usage using cookie files stored on the hard drive of all computers. The cookies contain valuable information that is transferred to your computers hard drive. In collecting this information we are able to improve our site and deliver a better and more personalised service. You can refuse to accept cookies but note that in doing this you may be unable to access certain parts of our site. You can do this by activating the setting on your browser which allows you to refuse the setting.

WHERE WE STORE YOUR PERSONAL DATA

Any data that we collect from you may be stored at, and transferred to, a destination outside the European Economic Area (EEA). This data may also be processed by staff that operates outside the EEA working us or one of our suppliers. In submitting your personal data, you agree to such a transfer and the storing and processing of this data. To ensure that your data is treated securely and in accordance with this privacy policy we will endeavour to take all reasonable necessary steps in protecting your privacy. We will do our best to protect your personal data, unfortunately, the transmission of information over the internet is not entirely secure and we cannot guarantee the security of data transmitted to our sites. Any transmission of data is at your own risk.

USES MADE OF THE INFORMATION

We use the information held about you in the following ways:
• To notify you of any changes to our service.
• To ensure that the content of our site is presented in the most effective manner for you and your computer.
• Where you have consented to be contacted, to provide you with the information, products and services requested from us or in the event that we feel will be of interest to you.
• To allow you to participate in interactive features of our service, should you choose to do so.
• In allowing ourselves to carry out our obligations concerning any contracts entered into between you and us. We may also use your data to inform you and provide information on goods and services that may be of interest to you.

DISCLOSURE OF YOUR INFORMATION

As defined in section 736 of the UK Companies Act 1985 we may disclose your personal information to any member of our group. These include our subsidiaries, our ultimate holding company and its subsidiaries. We may disclose your personal information to third parties:
• To the prospective buyer or seller of any business or assets in the event that we sell or buy.
• If we are under a duty to comply with any legal obligation to disclose or share your personal data. To protect the rights, property and safety of our customers or others, or to enforce our terms of use for the site. For the purposes of fraud protection and credit risk reduction this may include exchanging information with other companies and organisations.

THIRD PARTY WEBSITES

From time to time our site may contain links to and from the websites of our partner networks, advertisers and affiliates. If you visit any of these sites please note that they have their own privacy policies and you should check these before submitting any personal data. We cannot accept any responsibility or liability for these policies.

ACCESS TO INFORMATION

In accordance with the Act you can exercise the right to access any information we hold about you.

CHANGES TO OUR PRIVACY POLICY

We will where appropriate notify you by email of any changes made to our privacy policy and these changes will also be posted on this page.

CONTACT

We welcome any questions, comments and requests regarding this privacy policy and should be emailed to conference@naec.ge