Login
September 15, 2023Voiced by Amazon Polly |
Overview
Developers creating healthcare applications can incorporate AWS HealthScribe into their clinical software, enhancing their ability to deliver crucial visit highlights to medical professionals.
Example: 3M, ScribeEMR, and Babylon. In this blog, we introduced AWS HealthScribe. This HIPAA-eligible machine learning feature leverages speech recognition and generative AI to transcribe patient-clinician conversations and produce efficient clinical notes for healthcare applications.
Introduction
AWS offers advanced functionalities that empower the construction of high-performance, population-scale applications capable of storing, converting, and retrieving healthcare and life science data. Through tailor-made features, users can effectively oversee and extract insights from diverse healthcare and scientific data categories, encompassing clinical records, genomic and various omics data, medical imagery, and unstructured medical text and speech.
AWS HealthScribe integrates the functionalities, eliminating the necessity for integrating and fine-tuning distinct AI services. This capability allows you to accelerate the implementation process.
Pioneers in Cloud Consulting & Migration Services
- Reduced infrastructural costs
- Accelerated application deployment
Use Cases
- Accelerated documentation time – Facilitate clinicians in swiftly concluding clinical documentation using AI-generated clinical notes that are easily reviewable, adaptable, and ready for finalization within your application.
- Enhanced efficiency for medical scribes – Provide medical scribes with AI-generated transcripts and clinical notes, accompanied by consultation audio, to expedite the documentation turnaround time.
- Streamlined patient visit summaries – Create an interface enabling users to rapidly recapitulate vital conversational highlights within your application.
At present, AWS HealthScribe offers its support for the fields of General Medicine and Orthopedics.
JSON output files
An AWS HealthScribe job analyzes medical consultation to produce two JSON output files:
A Transcript file and a Documentation Insights file
1. Transcript file
In the Transcript file, AWS HealthScribe provides with
- Participant role identification allows differentiation between patients and clinicians within the conversation transcript.
- Transcript segmentation involves organizing dialogue sections within the transcript according to their clinical significance, such as casual conversation, subjective information, objective information, and more. This segmentation can be utilized to display specific segments of the transcript.
- Clinical entities encompass structured data like medications, medical conditions, and treatments referenced in the conversation.
2. Documentation Insights file
- The summary contains concise notes for significant portions of clinical documentation, including Chief Complaint, History of Present Illness, Assessment, and Plan.
- Evidence mapping establishes connections between each sentence within the AI-generated note summary and its source in the original consultation transcript. This facilitates users in verifying the accuracy of the summary within your application.
Initiating an AWS HealthScribe Job
AWS HealthScribe can be started with AWS CLI or AWS SDK
AWS CLI
1 2 3 4 5 6 7 8 | aws transcribe start-medical-scribe-job \ --region us-east-2 \ --medical-scribe-job-name first-medical-scribe-job \ --media MediaFileUri=s3://DOC-EXAMPLE-BUCKET/input-files/media-file.flac \ --output-bucket-name DOC-EXAMPLE-BUCKET \ --DataAccessRoleArn=arn:aws:iam::111122223333:role/ExampleRole \ --settings ShowSpeakerLabels=false,ChannelIdentification=true \ --channel-definitions ChannelId=0,ParticipantRole=CLINICIAN ChannelId=1,ParticipantRole=PATIEN |
Example:
With a request body and additional settings
1 2 3 | aws transcribe start-medical-scribe-job \ --region us-west-2 \ --cli-input-json file://filepath/first-medical-scribe-job.json |
First-medical-scribe-job.json file contains the below request body
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | { "MedicalScribeJobName": "first-medical-scribe-job", "Media": { "MediaFileUri": "s3://DOC-EXAMPLE-BUCKET/my-input-files/my-media-file.flac" }, "OutputBucketName": "DOC-EXAMPLE-BUCKET", "DataAccessRoleArn": "arn:aws:iam::111122223333:role/ExampleRole", "Settings": { "ShowSpeakerLabels": false, "ChannelIdentification": true }, "ChannelDefinitions": [ { "ChannelId": 0, "ParticipantRole":"CLINICIAN" }, { "ChannelId": 1, "ParticipantRole":"PATIENT" } ] } |
AWS SDK – Python (Boto3)
The AWS SDK for Python (Boto3) is used in the following example to start a start_medical_scribe_job request.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | from __future__ import print_functionimport timeimport boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "first-medical-scribe-job" job_uri = "s3://DOC-EXAMPLE-BUCKET/my-input-files/my-media-file.flac" transcribe.start_medical_scribe_job( MedicalScribeJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'DOC-EXAMPLE-BUCKET', DataAccessRoleArn = 'arn:aws:iam::111122223333:role/ExampleRole', Settings = { 'ShowSpeakerLabels': false, 'ChannelIdentification': true }, ChannelDefinitions = [ { 'ChannelId': 0, 'ParticipantRole': 'CLINICIAN' }, { 'ChannelId': 1, 'ParticipantRole': 'PATIENT' } ] ) while True: status = transcribe.get_medical_scribe_job(MedicalScribeJobName = job_name) if status['MedicalScribeJob']['MedicalScribeJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) |
Alongside a transcript, StartMedicalScribeJob requests also produce a distinct JSON-formatted documentation insights file. Both files are stored in the output location specified in your request.
- Transcript output
An AWS HealthScribe transcript file has the below format:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 | { "Conversation": { "ConversationId": "sampleConversationUUID", "JobName": "sampleJobName", "JobType": "ASYNC", "LanguageCode": "en-US", "ClinicalInsights": [ { "Attributes": [], "Category": "MEDICAL_CONDITION", "InsightId": "insightUUID1", "InsightType": "ClinicalEntity", "Spans": [ { "BeginCharacterOffset": 12, "Content": "pain", "EndCharacterOffset": 15, "SegmentId": "uuid1" } ], "Type": "DX_NAME" }, { "Attributes": [], "Category": "TEST_TREATMENT_PROCEDURE", "InsightId": "insightUUID2", "InsightType": "ClinicalEntity", "Spans": [ { "BeginCharacterOffset": 4, "Content": "mammogram", "EndCharacterOffset": 12, "SegmentId": "uuid2" } ], "Type": "TEST_NAME" }, { "Attributes": [], "Category": "TEST_TREATMENT_PROCEDURE", "InsightId": "insightUUID3", "InsightType": "ClinicalEntity", "Spans": [ { "BeginCharacterOffset": 15, "Content": "pap smear", "EndCharacterOffset": 23, "SegmentId": "uuid3" } ], "Type": "SYSTEM_ORGAN_SITE" } ], "TranscriptItems": [ { "Alternatives": [ { "Confidence": 0.7925, "Content": "Okay" } ], "BeginAudioTime": 0.16, "EndAudioTime": 0.6, "Type": "PRONUNCIATION" }, { "Alternatives": [ { "Confidence": 0, "Content": "." } ], "BeginAudioTime": 0, "EndAudioTime": 0, "Type": "PUNCTUATION" }, { "Alternatives": [ { "Confidence": 1, "Content": "afternoon" } ], "BeginAudioTime": 0.92, "EndAudioTime": 1.54, "Type": "PRONUNCIATION" }, { "Alternatives": [ { "Confidence": 0, "Content": "." } ], "BeginAudioTime": 0, "EndAudioTime": 0, "Type": "PUNCTUATION" }, { "Alternatives": [ { "Confidence": 0.9924, "Content": "You" } ], "BeginAudioTime": 1.55, "EndAudioTime": 1.88, "Type": "PRONUNCIATION" }, { "Alternatives": [ { "Confidence": 1, "Content": "lost" } ], "BeginAudioTime": 1.88, "EndAudioTime": 2.19, "Type": "PRONUNCIATION" }, { { "TranscriptSegments": [ { "BeginAudioTime": 0.16, "Content": "Okay.", "EndAudioTime": 0.6, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "SUBJECTIVE" }, "SegmentId": "uuid1" }, { "BeginAudioTime": 0.61, "Content": "Good afternoon.", "EndAudioTime": 1.54, "ParticipantDetails": { "ParticipantRole": "CLINICIAN_0" }, "SectionDetails": { "SectionName": "OTHER" }, "SegmentId": "uuid2" }, } ] } } |
- Documentation insights output
A documentation insights file has the following format:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | { "ClinicalDocumentation": { "Sections": [ { "SectionName": "SUBJECTIVE", "Subsections": [ { "SubsectionName": "HISTORY_OF_PRESENT_ILLNESS", "Summary": [ { "EvidenceMap": [ { "SegmentId": "uuid1" }, { "SegmentId": "uuid2" }, { "SegmentId": "uuid3" }, ], "SummarizedSegment": "The patient is seen today for a follow-up of weight loss." }, { "EvidenceMap": [ { "SegmentId": "uuid4" }, { "SegmentId": "uuid5" }, { "SegmentId": "uuid6" } ], "SummarizedSegment": "They report feeling more energy and craving carbohydrates, sugar, and fat." }, { "EvidenceMap": [ { "SegmentId": "uuid7" }, { "SegmentId": "uuid8" }, ], "SummarizedSegment": "The patient is up to date on their mammogram and pap smear." }, { "EvidenceMap": [ { "SegmentId": "uuid9" }, { "SegmentId": "uuid10" }, ], "SummarizedSegment": "Weight loss" } ] }, { "SubsectionName": "PLAN", "Summary": [ { "EvidenceMap": [ { "SegmentId": "uuid11" }, { ], "SummarizedSegment": "For the condition of Weight loss: The patient was given a 30-day supply of phentermine and was advised to follow up in 30 days." } ] } ] } } |
Conclusion
AWS HealthScribe generates and divides detailed conversation transcripts, distinguishes between patient and clinician speakers, extracts medical terminology, and creates initial clinical notes. By combining these functions, AWS HealthScribe streamlines implementation, lessening the necessity for integrating and enhancing individual AI services. The platform encourages responsible AI adoption for healthcare software vendors by linking AI-generated clinical notes to the original patient transcripts for each sentence. Security and privacy measures are inherently integrated into AWS HealthScribe, safeguarding sensitive patient data.
Drop a query if you have any questions regarding AWS HealthScribe and we will get back to you quickly.
Making IT Networks Enterprise-ready – Cloud Management Services
- Accelerated cloud migration
- End-to-end view of the cloud environment
About CloudThat
CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, AWS EKS Service Delivery Partner, and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.
To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.
FAQs
1. How does AWS HealthScribe collaborate with various other AWS services?
ANS: – Amazon Simple Storage Service (Amazon S3) can be employed to save both the input (audio files) and output (transcription and notes text files) produced by AWS HealthScribe. AWS HealthScribe leverages Amazon Bedrock’s extensive language model functionalities alongside Amazon Comprehend Medical and Amazon Transcribe Medical to create refined clinical notes through comprehension of patient-clinician interactions.
2. Does AWS HealthScribe provide support for all medical specialties?
ANS: – AWS HealthScribe is fine-tuned for the domains of general medicine and orthopedics.
WRITTEN BY Abhilasha D
Abhilasha D is a Research Associate-DevOps at CloudThat. She is focused on gaining knowledge of Cloud environment and DevOps tools. She has keen interest in learning and researching on emerging technologies.
PREV
Comments