Query with Llama 3

• Ask Earth: Retrieves answers to users’ questions.
• Explore: Gathers information about a selected location on the globe and generates short descriptions for landmarks.
• Today in history: Identifies a significant historical event that occurred on the current month and day according to the user’s local time.
• Daily quiz: Creates trivia questions of varying difficulties, including incorrect options and the latitude/longitude coordinates for the correct answer’s location.

/services
    /llama
        QueryLlamaService.kt
        IQueryLlamaServiceHandler.kt


        /models
            BedrockRequest.kt
            BedrockResponse.kt
            OllamaRequest.kt
            OllamaResponse.kt

fun submitQuery(
    query: String,
    creativity: Float = .6f, // temperature
    diversity: Float = .9f,  // top_p
    handler: IQueryLlamaServiceHandler
) {
    if(queryTemplate.isNullOrEmpty()) {
        throw Exception("Llama query template not created")
    }


    val fullQuery = String.format(queryTemplate!!, query)
    val temperature = creativity.clamp01()
    val top_p = diversity.clamp01()


    val serverType = SettingsService.get(
        KEY_LLAMA_SERVER_TYPE, LlamaServerType.AWS_BEDROCK.value)
    when (serverType) {
        LlamaServerType.OLLAMA.value -> queryOllama(
            fullQuery,
            temperature,
            top_p,
            handler
        )


        LlamaServerType.AWS_BEDROCK.value -> queryAWSBedrock(
            fullQuery,
            temperature,
            top_p,
            handler
        )
    }
}

val jsonMediaType = "application/json; charset=utf-8".toMediaTypeOrNull()
val nativeRequest = OllamaRequest(query, OllamaRequestParams(temp, top_p))
val requestBody = gson.toJson(nativeRequest).toRequestBody(jsonMediaType)


val request = ollamaRequestBuilder.post(requestBody).build()

// Embed the prompt in Llama 3's instruction format.
val instruction = """
    <|begin_of_text|>
    <|start_header_id|>user<|end_header_id|>
     <|eot_id|>
    <|start_header_id|>assistant<|end_header_id|>
""".trimIndent().replace("", query)


val nativeRequest = BedrockRequest(instruction, temp, top_p)
val requestBody = gson.toJson(nativeRequest)


val request = InvokeModelWithResponseStreamRequest {
    modelId = "meta.llama3-8b-instruct-v1:0"
    contentType = "application/json"
    accept = "application/json"
    body = requestBody.encodeToByteArray()
}

• temperature: Controls the creativity level of the model. A low value of 0.1 ensures minimal randomness and higher predictability.
• Top_p: Determines the diversity level of the model. The default value of 0.9 enhances response diversity.

• explorescreenbase_query: In Explore play mode, geocoordinates formatted in common notation (E/W and N/S instead of +/- to denote hemisphere) are injected at token 1. The place name returned from the Google Geocoding API (if found) is injected at token 2. For more information about the Google Geocoding API, see Geocoding API Overview.
  Example: “What is one notable city or landmark near the coordinates 37.4°N, 139.76°E in Japan?”
• today_in_history_base_query: In Today in History play mode, the user’s local date is injected at token 1 in the format MMMM d.
  Example: “What is one notable event in history that occurred on Aug 30?”
• base_query_template: All queries are injected into this base query template in the QueryLlamaService.submitQuery function before being sent to the model server. This query accomplishes two tasks: it limits response length (preventing overflow in the allocated panel area and eliminating scrolling), and it transforms the response into Markdown format. All Llama text responses are displayed inside the MarkdownText composable function from the compose-markdown dependency. This approach effectively displays nicely formatted responses from Llama in the application.
  User question example: “In a short response formatted with markdown, answer the following question: Where are the tallest mountains on Earth?”
  Today in History example: “In a short response formatted with markdown, answer the following question: What is one notable event in history that occurred on Aug 30?”

val fullQuery = String.format(templateQuery, data)


QueryLlamaService.submitQuery(
    query = fullQuery,
    creativity = 1f,
    diversity = .9f,
    handler = object : IQueryLlamaServiceHandler {
        override fun onStreamStart() {
            // (optional) hide loading message/graphic
        }


        override fun onPartial(partial: String) {
            // (optional) update result UI with partial response
        }


        override fun onFinished(answer: String) {
            // update result UI with full, final response
        }


        override fun onError(reason: String) {
            // handle querying error
        }
    }
)

• Daily quiz: Questions and answers were generated with the following instructions:
  Generate 100 trivia questions related to Earth geography and cultures, ranked from easy to difficult, including the latitude and longitude coordinates of each location answer. Format the response in XML and provide two incorrect answers for each question.
• Explore: Landmark descriptions were generated with the following instructions:
  Provide short descriptions for each of the following landmarks: the Great Egyptian Pyramids, the Eiffel Tower, Chichén Itzá, the Sydney Opera House, Taj Mahal, the Christ the Redeemer statue, the Colosseum, Mount Vinson, and Victoria Falls. Format the responses in XML, including the name, description, latitude, and longitude of each landmark.