Integration in theory and practice. Prof. Pat Bazeley

Integration of varied approaches to data collection and/or analysis is an essential defining feature of mixed methods research. Integration has been defined as purposeful interdependence between methods within a project, with the expectation that benefits will flow from it. Integration occurs through iterative exchange during the conduct of a project, and through planning of one or more ‘points of interface’ between methods – points at which different approaches are deliberately brought together.
Coherent design supporting integration involves consideration of common purposes and paradigms as well as planning for the practicalities of sequencing, sampling, data sources, and teamwork. Coherent analysis supporting integration gives consideration to all available data, combines data and analyses in variations on one or more of six ways, and will become especially evident in the presentation of the results, discussion and conclusions of a study.
This presentation will unpack the concept of integration, with its implications for practice in a mixed methods environment.

Pat Bazeley is Director of Research Support P/L and Adjunct Professor in the Translational Research and Social Innovation Centre at Western Sydney University. Since graduating in psychology she has worked in community development, project consulting and in academic research development. For almost 30 years Pat has been providing research training and project consulting to academics, graduate students and practitioners representing a wide range of disciplines across Australia and internationally. Her particular expertise is in helping researchers to make sense of qualitative, survey, and mixed methods data, and to use computer programs for management and analysis of data. Pat’s research has focused on qualitative and mixed methods data analysis, the development and performance of researchers, and the wellbeing of older women. She has published books, chapters, and articles on mixed methods and qualitative data analysis. She serves on the Editorial Boards of the Journal of Mixed Methods Research and Qualitative Health Research, and was 2015–2016 President of the Mixed Methods International Research Association.

Mixing it up: Crossing cultures, breaking barriers. Assoc Prof Caryn West

Understanding varied human experiences in an increasingly integrated world is fundamental in ensuring the social and cultural hallmarks that define a civilized nation are not lost. Mixed methods is a particularly useful tool when researching cross cultural and cultural experiences and related phenomena. Incorporation of community participation into mixed method approaches, such as developing community partnerships, engaging cultural advisory boards and creating knowledge mobilisation plans increases the effectiveness and prospective benefits of the research process and its results. This is especially true for historically and contemporarily marginalised cultural groups. Mixed method approaches provide the framework for culturally and contextually grounded models which are applicable to the culture the model intends to serve.

A/Prof Caryn West has built a diverse research portfolio within nursing and public health through her professional roles and awards, including an early career fellowship with the National Health and Medical Research Council (NHMRC) and directorship of the World Health Organisation Collaborating Centre (WHOCC) for Nursing, Midwifery Education and Research Capacity Building. Caryn’s multiple areas of research interest include disaster management, nursing education, and alcohol-related injuries in Australian Indigenous communities which are drawn together through the theme of resilience.
Caryn maintains a strong international reputation through her involvement with research centres, international humanitarian projects and international mixed methods research associations.
Working extensively with Asia Pacific and Western Pacific countries A/Prof West is the current executive committee member for the Western Pacific Region of the Global Network of World Health Organisation Collaborating Centre for Nursing and Midwifery, and a committee member and advisor for the Asia Pacific Emergency Disaster Nurses Network (APEDNN).

Exploring Methodological Boundaries: Rotating five vantage points about mixed methods research. Prof. Elizabeth Creamer

The diversity of vantage points expressed by leading figures that shaped the development of mixed methods as a distinct methodology is often a source of both confusion and frustration to newcomers to the field. From conceptualizing a study through to conducting the analysis and drawing conclusions, each of the perspective highlights a different aspect of the research process. Another way to view the pluralism is as an indicator of the on-going vibrancy in this far-ranging community of scholars that extends across disciplines, content areas, and continents. My aim is to foreground the agency and innovation that the diversity of perspectives offers to researchers and practitioners as they enter the field and tailor their research to their own situated epistemology.

Dr. Elizabeth Creamer is Professor Emerita Educational Research and Evaluation in the School of Education at Virginia Polytechnic Institute and State University (U.S), where she served in a wide variety of faculty and administrative roles. Creamer is a research methodologists who taught graduate level research methods courses for more than twenty years in mixed methods and qualitative approaches to grounded theory. Prior to that she was a faculty member in the Women’s Studies program, teaching courses in feminist research methodology and theory. She is the author of the 2018 SAGE textbook, An Introduction to Fully Integrated Mixed Methods Research that introduces a framework for mixing across all phases of the research process. Under contract with Routledge, she is in the process of writing a new textbook, Advancing Grounded Theory Development with Mixed Methods Research. Over the course of her 35-year career, Creamer authored 4 books or monographs, 127 journal articles and book chapters, and 95 conference presentations. She lead workshops about mixed methods in diverse settings, including in Vienna, Austria and Japan. Elizabeth is serving as the fifth president of the Mixed Methods International Research Association (MMIRA).

Workshops: Wednesday

Workshop One: Editorial Workshop Panel
Panel: The Editorial Workshop will be presented as a Panel of those senior MMR academics who currently have roles as Journal Editors or have in the past.

Targeted attendees: Anyone that wants to submit journal submissions based on a MMR study or a MMR methodology papers

Key issues will be discussed in terms of what makes for a successful submission and what the Editors are looking for in submissions. Key pitfalls of submissions will also be addressed.

Workshop Two: Heart & Soul of Mixed Methods: The Importance of Culture     
Presenter:  Assoc Prof Caryn West
Targeted attendees: Novice researchers, Cross cultural researchers 

As researchers we know that each culture is unique.  Differences exist and they are easily quantified and described: country, region, language, traditions, beliefs, religion, family and so on.  In fact we have become so good at identifying the differences in cultures that we often overlook things.

“Sometimes the questions are complicated and the answers are simple.” Dr Seuss

Although mixed methods is a particularly useful when researching cultural experiences related to phenomena too often we fall short of our responsibility to be truly culturally inclusive. From the first notion of an idea, scholarly research is informed by a philosophical stance. Captured within that philosophical stance are the researcher’s beliefs, values and assumptions, in essence their view of reality.  Historically in academia a Western approached to research has been employed, which although a suitable blueprint for processes, lacks the innate insight Indigenous paradigms inherently bring to the research.
In order to unfold the rich layers of a culture, researchers need t be consciously and continuously aware of culture and what drives it. Above and beyond the simplistic differences of country, region, language and traditions the very essence of culture in reflected in the history of a people. It is in the arts, our collective heritages, our local and national economies, and in the everyday objects we value and use.  It is a sense of belonging, of wellness and learning, and of growth and healing.

This workshop aims to explore the importance of cultural engagement throughout the research process and how this can be reciprocally beneficial for both researcher and the community involved

Workshop Three: Creating Joint Displays to Facilitate Integration: A Step-by-Step Approach
Presenter: Dr Timothy Guetterman
Targeted attendees:
The workshop is intended for any individuals (faculty, staff, students, other researchers) who have some foundational understanding of mixed methods. Because the workshop includes a review of integration strategies for major mixed methods designs, it is appropriate for beginners. It will be particularly helpful to: 1) anyone planning a mixed methods study to ensure meaningful integration, 2) individuals who are currently conducting a mixed methods study as a framework to think about integration, and 3) those writing a mixed methods
manuscript as a way to represented integrated analyses.

Integration of qualitative and quantitative methods is the key feature of mixed methods research that distinguishes it from other approaches. However, integration remains somewhat elusive and a challenge for researchers. Recent literature, such as the Bazeley (2018) textbook address integration squarely and tout its importance and value to mixed methods. Nevertheless, researchers need applied and hands on guidance in
thinking about integration. One approach that provides a framework to think about integration are joint displays. Joint displays are a way to visually integrate qualitative and quantitative results and to represent integration in proposals, manuscripts, and dissertations (Guetterman, Creswell, & Kuckartz, 2015; Guetterman, Fetters, & Creswell, 2015) . Joint displays have emerged as a state-of-the art procedure in mixed methods research. Drawing from numerous disciplines, the beginning of this workshop will review three procedures for achieving integration and discuss exemplar joint displays connected to procedures for integration and the overall mixed methods design. The majority of the workshop is hands on and will focus on selecting an appropriate joint display and creating a mock joint display. We will review template joint displays to assist with their construction. Even for researchers at the planning stage, envisioning joint displays is useful to ensure meaningful integration.

Bazeley, P. (2018). Integrating analyses in mixed methods research. London, UK: SAGE.
Guetterman, T., Creswell, J. W., & Kuckartz, U. (2015). Using joint displays and MAXQDA software to represent the results of mixed methods research. In M. McCrudden, G. Schraw & C. Buckendahl (Eds.), Use of visual displays in research and testing: Coding, interpreting, and reporting data (pp. 145-176). Charlotte, NC: Information Age Publishing.
Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative results in health science mixed methods research through joint displays. The Annals of Family Medicine, 13(6), 554-561. doi: 10.1370/afm.1865

Timothy C. Guetterman, PhD, is an applied research methodologist specialized in mixed methods research.  He writes empirical and methodological articles and books to advance rigorous methods of quantitative, qualitative, and mixed methods research. Tim’s methodological research focuses on integrating qualitative and quantitative research using visual joint displays and intersecting qualitative designs with mixed methods research. Tim’s empirical work uses mixed methods research to investigate the use of technology in health, health education, and educational assessment. 

Workshop Four: Mixed Methods Data Analysis using a Web Based Application
Presenters:Dr Michelle Salmona Institute & Prof. Dan Kaczynski Institute for Mixed Methods Research

Targeted attendees: Faculty, graduate research students

Researchers are increasingly adopting the use of data analysis software and computer applications for mixed methods studies. This workshop will discuss the impact of new developments in technology and the use of Dedoose, a cloud-based data application, to identify, manage and analyse qualitative and mixed methods data. An overview of analysis strategies will be woven throughout the seminar. Discussion will highlight the influence that research design decisions have upon the methodology and, ultimately, the quality and credibility of a study.
There will be activities and exercises for participants to gain practical hands-on skills using this innovative mixed-methods application which promotes collaboration and teamwork, and the production of trustworthy results and findings. This hands-on interactive workshop will feature the use of Dedoose as a technological tool to help researchers manage and analyse a growing range of social science data.
The workshop will also explore and discuss strategies for multiple members of a research team to work simultaneously, in real-time, from any Internet connected device. Workshop participants will be encouraged to discuss team strategies which improve the quality and efficiency of analysis results and presentation of findings.
Participants are encouraged to bring a laptop and digital data to work with during the workshop. One month’s free access to Dedoose will be available for new users during the workshop (no downloaded software required). Just turn up with your laptop. There will be ample opportunity to try out various features within the cloud. Having the use of a laptop or tablet during the workshops will enhance participants’ overall learning experience.

Professor Dan Kaczynski is a Senior Research Fellow at the Institute of Mixed Methods Research (IMMR).  He is currently appointed as a Professor supervising doctoral candidates at the University of Canberra, Australia and is Professor Emeritus at Central Michigan University.  Dan’s research interests in the United States and Australia advance technological innovations in qualitative and mixed methods data analysis in the social sciences.
Dan is actively engaged as a program evaluation consultant and has over 20 years’ consulting experience conducting state, national and international evaluations.  He has held leadership roles in program administration and as a research center director with extensive experience as Principal Investigator of over $35 million in grant awards.  His work has been shared professionally with over 230 professional presentations nationally and internationally.  Dan has written over 40 published research articles and co-authored six books and book chapters.  In addition, Dan has supervised over 95 doctoral dissertations and professional specialist thesis. 

Dr. Michelle Salmona is President of the Institute for Mixed Methods Research (IMMR) and an Adjunct Associate Professor at the University of Canberra. With a background as a project management professional and a senior fellow of the Higher Education Academy, UK, she is a specialist in research design, methods and analysis.  Michelle is a co-founder of IMMR, building global collaborations for grant development, and customized training and consultancy services for individuals and groups engaged in mixed-methods and qualitative analysis.

Michelle works as an international consultant in program evaluation; research design; and mixed-methods and qualitative data analysis using data applications.  Her research focus is to better understand the research process; and build data-driven decision making capacity in the corporate world, and she is particularly interested in the relationship between digital tools and doctoral success.  Michelle is currently working on projects with researchers from education, information systems, business communication, leadership, and finance.

Workshops Thursday

Workshop Five: Analysing and integrating mixed methods data    
Presenter: Prof Pat Bazeley

Targeted attendees: Anyone doing MM research, but realistically, probably best for those who already have some background knowledge of MM design and practice.

Analysing mixed methods data is more than just comparing and perhaps melding the conclusions drawn from separate data components. Integration, which lies at the heart of mixed methods research, is about bringing varied data and analysis strategies together in multiple ways throughout a project – as iterative exchange, and also at one or more deliberate ‘points of interface’ – so that the varied approaches taken become interdependent in achieving a common theoretical or research outcome. This workshop will present a range of complementary, comparative, relational, transformative, and visual processes for integrating varied data sources and strategies for analysis, illustrated through reference to a variety of mixed methods studies from multiple disciplines. These integrative processes and strategies are designed to ensure that data and available tools are used most effectively in building strong and useful conclusions that contribute to meeting the purpose of the research.

Key text: Bazeley, P. (2018). Integrating analyses in mixed methods research. London: Sage

Workshop Six: Quality Criteria in Mixed Methods Research
Presenter: Assoc Prof Ros Cameron

Targeted attendees: The workshop is a useful resource for novice researchers and Early Career Researchers (ECRs) embarking on MMR studies. It is also useful and expedient resource for academics and researchers who are required to review MMR journal submissions, examine MMR theses and judge research grant applications applying MMR designs. It is also a valuable resource for those teaching MMR or conducting research training workshops and advanced research capacity building.

  • Introduce notions and language of quality criteria
  • Overview of quality criteria in QUAN research
  • Explore the Quality criteria debates in QUAL research
  • Introduce the development and reviews of quality criteria in MMR
  • Describe the use of critical appraisal frameworks in Systematic Mixed Studies Reviews  (SMSRs) 

Associate Professor Roslyn Cameron (PhD) is Head of Discipline HRM/Management and Director of Research & RHD at the Australian Institute of Business, Adelaide, South Australia. She is a Fellow of both the Australian Human Resources Institute (FAHRI) and the Australian Institute for Training and Development (FAITD) and is Co-Convenor of the Mixed Methods Research Special Intertest Group of ANZAM. Ros has just been elected to the Board of the Mixed Methods International Research Association (MMIRA). She has been the recipient of several large-scale workforce development research grants and an array of smaller scale research grants related to skilled migration, workforce readiness/employability and future skilling totalling $1.2m. She has over 90 publications.

Workshop Seven:  Fully integrated Mixed Method Research approaches
Presenter: Prof Elizabeth Creamer

Targeted attendees: Anyone doing Mixed Method Research

This interactive workshop introduces fully integrated mixed method research (FIMMR) as a methodology for integrating qualitative and quantitative approaches throughout the research process from the way it is initially conceived through the way it is executed.
Learning goals to explore ways that multiple sources of data can be engaged dialectically in (a) the construction of research questions, (b) the development of a sampling plan, (c) during data collection and analysis, (d) leveraging dissonance for theoretical insight, and (d) in the drawing of conclusions to create an original explanatory framework.
Participants completing the workshop will come away with a list of creative ways the qualitative and quantitative data can be integrated, particularly during analyses.

Workshop Eight: Quantitative Methods in Indigenous Educational Contexts: using Legitimation Code Theory in the interfaces between quantitative, qualitative and kaupapa Māori research.
ers:  Dr Brian Tweed and Dr Pania Te Maro

Targeted attendees:  Researchers interested in either legitimation code theory and/or quantitative methods research in indigenous contexts.

This workshop will engage attendees in the tensions surrounding the use of quantitative methods research in indigenous contexts focussing on kaupapa Māori research. The practical work will use Legitimation Code Theory to illustrate how non-epidemiological studies which collect qualitative data may be quantified and used to generate measures of the strengths of ‘legitimation codes’ implicit in pedagogical practices. It will be argued and, it is hoped, demonstrated that quantitative methods in indigenous contexts is both possible and productive as long as great care is taken to attend to context, people and purpose.

Originally from the UK, Brian has worked extensively as a teacher and advisor in both English-medium and Māori-medium schools in New Zealand for the past 30 years. He completed his doctorate at Victoria University in 2016 examining the struggle of Māori immersion schools with mathematics education. He is currently a senior lecturer in the Institute of Education at Massey University.
Brian’s research interests include critical realist philosophy and social research, the sociology of knowledge and education, and emancipatory pedagogies especially from a non-indigenous perspective. He also has a special interest in mixed methods research with a focus on the ontological commensurabilities between qualitative and quantitative approaches.

Ko Hikurangi te maunga, ko Waiapū te awa, ko Ngāti Porou te iwi, ko Te Kapa a Hinekōpeka te tūrangawaewae, ko Pōkai te wharenui, ko Te whānau ā Pōkai te hapū, ko Ihipera rāua ko Harawira oku tīpuna, ko Fred taku pāpā, ko Margaret taku māmā, ko Pania Te aro ahau!

Pania will be graduating with her PhD in March this year! It’s called, “Mai i ngā rā o Mua: dialectical and knowledge-power relations in the interactions of Kura with maths education.” It asks and goes some way to answering the question, “How is it that maths education interrupts Kura communities’ commitment to Kaupapa and Mātauranga Māori; and has power to format Māori identities and expectations of education? Pania has used kaupapa Māori research theory along with Marx and Foucault to interpret data. She can see a day when Kura are able to deliver beyond-curriculum-maths, and do so in a way where kaupapa and mātauranga Māori are equitably privileged with kaupapa and mātauranga mathematics (not just curriculum maths, but maths that supports deeper more conceptual understandings and mathematical principles).
Pania’s teaching background is from Kōhanga to Wānanga in mainly kaupapa Māori AND a couple of kaupapa English educational settings.  Her research is mainly in mathematics in the compulsory sector, but she has also worked with and carried out research about adult numeracy as this is also a field that she really enjoys and want to make a difference in.

Using Software for Mixed Methods

Free one-hour demonstrations of the following software products designed to assist mixed methods researchers

Innovations in Technology – Using Dedoose for Data Management & Analysis
Dr Michelle Salmona

Dedoose is a cross-platform cloud-based mixed-methods application which facilitates the coding and analysis of qualitative and mixed methods data and their integration with demographic and other quantitative data. Collaboration is one of the key drivers behind the development of Dedoose, allowing project owners to provide others with access to their projects, facilitating real-time online team engagement.

Using NVivo  for mixed methods analysis
Prof Pat Bazeley

NVivo offers a range of tools to meet the needs of mixed methods researchers. Its particular strength is in management of complex data, for example where multiple whole or part sources need to be considered together for particular cases, and also linked with demographic and variable data. NVivo allows for inclusion of a wide range of data types within a project, has a flexible coding system, and provides versatile tools (crosstabs and matrix analysis) to support complementary and comparative analyses that combine text and numeric data. Qualitative coding information can be exported as variable data (in Excel or SPSS format) for further statistical analysis. Mapping and visual analysis tools additionally provide new ways of visualizing data and the relationships within data.

Using MAXQDA for support of MMR
Tim Guetterman

MAXQDA has always had a focus on supporting mixed methods research. Text and visual documents can be imported and linked to associated demographic and variable data. Coding is straightforward, with codes able to be converted to variable data for use in comparative analyses or for export to Excel. A flexible memoing system and tools for summarising and visualising data in a variety of ways are strengths of the software. MAXQDA provides a set of tools specifically designed to support integrative mixed methods analyses – supplementing crosstabs and matrix displays with an interactive comparative quote matrix, joint displays of text and statistical data, and summary grids. Basic statistical analyses (e.g., chi, t, ANOVA) can be carried out within the software using coding and/or variable data, and there are mapping and other visual tools that support both analysis and presentation of results.