Research on an Online Community providing psychological support
SI 529 Online Community
Sili Jiang / Yu Qin
How do users perceive the community in term of helpfulness in general?
Background
The Forum AgingCare Forum(https://www.agingcare.com/caregiver-forum) is an online community tailored to family caregivers. Users share and discuss challenges or confusions that they encounter during their caregiving experiences and get active responses from other users or even experts in as quick as 10 mins. This forum ” , “Alzheimer”, “Family Relationship”. Each section is further divided into “Q & A” and “Discussions.
The Users Users in this community are categorized into two types—caregivers and experts.
-
Caregivers: Users as caregivers can post their questions into sections that are related to their questions in either “Q & A” or “Discussions”, though there is no rule banning cross-posting. Caregivers are able to switch between seeking help and giving help freely not only within the community but also spreading their influence to members outside this community.
-
Expert - The expert members are usually certified professionals in caregiving consultant, financial planning, Medicaid asset protection planning, and legal issues. They also respond to some posts with suggestion and their responses are usually given a special mark to be distinct from family caregivers.
Motivation
We are interested in online communities which provide support for people experience emotional difficulties, relationship tensions and other stressful events in their daily life.
-
Whether do members in this seemingly highly engaging and supportive community get the support they want?
-
How do they perceive the level of helpfulness of the support they have received?
Goal
-
Hack into the concept of helpfulness to evaluate whether such community satisfies the members’ needs to be supported.
-
Apply our findings to improve the design of this and other online communities and to make them effective support tools for people experiencing emotional difficulties, stressful events or other challenges in their daily life.
Research Questions
Overreaching Research Question:
(Helpfulness)-How do users perceive this community in term of helpfulness in general?
To answer this overreaching questions, we need to answer two basic questions first.
FRQ1: (Scenarios) What are the typical phenomenon/situations that (family) caregivers face?
FRQ2:(Interactions) How do users interact with each other in this community?
-
Users who post their questions; users who give comments and support under posts/experts who give out advice.
-
Users give comments, answers, hugs, likes and think an answer helpful.
Based on the previous two questions, we rephrased the overreaching research questions in a more specific way:
→ How do users with different living situations and facing different problems perceive this community in term of helpfulness in general?
→ How does the perception of helpfulness vary according to different user interaction patterns?
Research Process
1️⃣ Stage one: Observations
Method 20 hours observations
Sili and I individually conducted 2 hours observation / per day —approximately 20 hours’ observation during Feb 24--Mar 7 continuously.
Goal
Grasp how the community works from both broad and detailed perspectives to scope the project and construct analysis framework for answering complicated research questions.
-
Broad: how does the website work in terms of different sections and topics
→ find the most interesting and valuable part to do further analysis
-
Detailed:
-
how do active users behave
-
how do active posts and their responses look like
-
→ Extract factors that can describe the type of users and interaction patterns to construct analysis framework.
Process
-
Pick a proper time to observe: We conducted our major observation during late hours at night, usually between 9 pm--11 pm since the end of a day provided a good opportunity to observe all happenings over the day. But we were surprised by the activity level of this community that users still update around midnight that we occasionally stayed as late as 11 pm-- 1 am. Each time the observation lasted for 30 mins to 2 hours. We did long-hour observation on the first 2 days in order to grasp how the community works
-
The Observation Content & Log Sheets Two team members conducted observation from the following three levels, we put out takeaways from observations into three well-structured log sheets
△User Level:
As for members, they first captured our attention as their posts expressed some opinion or experience that stood out or received most heart shapes under original posts. Then we checked their profile pages in order to observe their activities in this community. We also differentiated them along user types whether he/she is an expert or not.
△Post and Review Level:
For example, we kept note of posts on its content, the interaction between members and some special context such as expression related with “thank you” as we believe that it is an indicator that members found some comments helpful.
△Community Level:
As mentioned, AgingCare Forum has been divided into sections to address typical situations in caregiving. We then spend some time observing the performance of these 48 sections in both “Discussion” and “Q&A” and summarized our findings listed in the table below. We tracked these sections mainly on two dimensions--last updating time and the number of replies that original posts have received. It turned out some sections have been really active while some sections have been really silent that the last update time aged years ago or even received none following-up replies at all.
Output
→ find the most valuable part in terms of helpfulness to do further analysis
Clustered these 48 sections into 12 categories as listed below in terms of content similarities and the level of activeness based on background research on aging and caregiving as well as observations of this community
We adopted the reasoning of affinity wall and narrowed these 48 sections into 12 categories as listed below and we highlighted sections that have been popular in this community from the perspectives of most recent update time and the number of replies that a post has received.
→ Extract factors that can describe the type of users and interaction patterns to construct:
Data Collection Metrics
the Codebook
2️⃣ Stage Two: Content Analysis
The second stage: we focused on the most valuable part in terms of helpfulness
we identified from the observation to do the future analysis.
Method Conventional content analysis & Summative content analysis
Why?
Such community highly relies on text-based interactions to satisfy users’ needs. Thus, if we want to understand how users perceive this community in term of helpfulness, we need to hack into the word usage in every post and response. The qualitative content analysis combined with quantitive content analysis could be a great choice to address our research questions.
Goal
Find the relationship between factors we extracted from the first stage to address the research questions:
→ How do users with different living situations and facing different problems perceive this community in term of helpfulness in general?
→ How does the perception of helpfulness vary according to different user interaction patterns?
Process
-
Theme generate: Focused the 13 representative sections clustered from 48 sections based on the level of Activeness during our observation for further analysis.
-
Data collection
-
Included highlighted sections and at least one section from 13 categories for our further content analysis.
-
Used factors we extracted from the first stage that can address the first 2 research questions we proposed--different caregiver situations and interactions between members in this community to collect the following data through a customized Web Crawling Python program.
546 (6x7x13) responses including original posts were captured for our study.
3. Code the post and responses
-
Based on the first stage observation and background research, we developed our codebook.
-
To understand our first research question— what scenario these caregivers have experienced, we focused on the following aspects in our codebook:
* What are these caregivers motivations’ in posting to this community? 2 teammates marked all posts were motivated to seek solutions or to initiate discussion
* What role this poster has played in the caregiving situation that he or she described.
* What’s the situation of care recipient(s)?
-
To understand our second research question—what interaction pattern members have in this community, we focused on the following aspects in our codebook:
* Whether an original poster continued posting
* if there was any “name-calling” behavior.
* Some certain words like repeated “thank you” or appreciation as a metrics of helpfulness.
* opinions and emotions in these replies.
-
What’s more, for the quantitive part:
* count how many users have involved in this post
* count the number of users involved in this post indicates how much attention and support that an original poster has received.
* count how many replies from same users
4. Data Analyze:
-
Define pattern A, B, C based on the code
-
Connections between A - B - C
-
Quote to support pattern
Output
→ Find Relationship between factors: the posts with different motivations, problems types have different interaction patterns, which cause different understanding of helpfulness.
Findings and Recommendations
FRQ1: What are the typical phenomenon/situations that (family) caregivers face?
*Finding 1 Posts in the same topic tend to share similar motivations
*Finding 2 Nine Typical Scenarios/Themes in AgingCare Forum
We created a tree diagram to organize and analyze the caregiver situations that we found via coding these posts. We came up with the following 9 main themes which can represent typical scenarios to some extent.
FRQ2: (Interactions)-How do users interact with each other in this community?
*Finding: Interaction patterns differ in terms of different motivations behind the posts.
RQ1: How do users perceive the helpfulness of the community?
*In summary, we believe posts which considered helpful have these feature: *long activation period, updates from original posters, responses from different users; other users’ multiple replies to one and number of heart-shapes given to replies.
We think this online community is helpful to a member if his or her needs are met and member’s motivations can be used to assess their needs. If their needs were met from responses, we noticed they were more likely to express appreciation or the feelings of “not being alone”. We were limited in time and labor force to finish summative content analysis regarding helpfulness measure though we grasped the context where “thank you” or “appreciation” appeared.
Improvement
We conducted coding manually, which cost us a lot of time and energy, but we can use some computational techniques like Natural Language Processing or Machine Learning to automatically process the text.
