, and in body.\nEnsure the use of a modern and stylish HTML design in the body display by using inline style = '', \nHighlight the body into a stylish and modern
, \nClearly Mention and Highlight the platform name: ALPHATX with stylish text decoration using inline style.\nHighlight the platform name: IPTV30SAVE at checkout for 30% off with stylish text decoration using inline style.\nHighlight the first CTA: by using the discount code IPTV30SAVE at checkout for 30% off when purchasing our premium membership at https://xtreamtech.net/premium-membership and unlock all premium IPTV subscriptions simultaneously. with a proper anchor texts into a new line centered HTML button tag. \nHighlight the second CTA: by joining our facebook group : Free IPTV Codes https://iptv2live.com/fb.php to get the latest trial IPTV codes with a proper anchor text into a new line centered HTML button tag. \nAvoid keyword stuffing; focus on creating valuable, informative content for the reader.\nMake sure the product description is engaging, informative, and optimized for search engines while providing value to the reader.\nMake sure to never mention the word \"Call to Action\" in your response.\nFinally make sure that the output overall pass all AI test detection by using Humanized and natural Text writing style."},{"assistant_initiates":false,"asst_text":"To create a more chatbot-like interface using Gradio, you can make adjustments to enhance the interactive user experience and visibility. Here are some suggestions to make the Gradio-based chatbot experience more intuitive and engaging:\n\n1. **Add User and Assistant Avatars**: You can use avatars to distinguish between user and system messages visually.\n\n2. **Improved Layout**: Adjust the layout for better accessibility by reorganizing UI elements.\n\n3. **Customize with CSS**: Use CSS to provide visual cues and a more polished look.\n\n4. **Streamlined Message Handling**: Ensure messages are handled smoothly for both input and output.\n\nHere\u2019s how you can start implementing these suggestions in your code:\n\n```python\nimport os\nimport time\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer, BitsAndBytesConfig\nimport gradio as gr\nfrom threading import Thread\n\nMODEL_PATH = r\"C:\\Users\\shermanhonsf\\Desktop\\Python\\llama_develop\\model\\Llama3.2-1B-Instruct-HF-Model\"\n\nTITLE = \"\"\"\n# Llama 3.2 1B Chatbot\n\"\"\"\n\nPLACEHOLDER = \"Hi, I'm Llama. Ask me anything.\"\n\nCSS = \"\"\"\n.duplicate-button {\n margin: auto !important;\n color: white !important;\n background: black !important;\n border-radius: 100vh !important;\n}\nh3 {\n text-align: center;\n} \n.chat-interface {\n background-color: #f5f5f5;\n overflow-y: auto;\n}\n.chat-message.user {\n text-align: right;\n margin: 5px 0;\n}\n.chat-message.assistant {\n text-align: left;\n margin: 5px 0;\n}\n\"\"\"\n\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\nquantization_config = BitsAndBytesConfig(\n load_in_4bit=True,\n bnb_4bit_compute_dtype=torch.bfloat16,\n bnb_4bit_use_double_quant=True,\n bnb_4bit_quant_type=\"nf4\"\n)\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)\nmodel = AutoModelForCausalLM.from_pretrained(\n MODEL_PATH,\n torch_dtype=torch.bfloat16,\n device_map=\"auto\"\n)\n\nmodel.to(device)\n\ndef stream_chat(message: str, history: list, **kwargs):\n conversation = [{\"role\": \"system\", \"content\": kwargs.get('system_prompt', 'You are a helpful assistant')}]\n \n for prompt, answer in history:\n conversation.append({\"role\": \"user\", \"content\": prompt})\n conversation.append({\"role\": \"assistant\", \"content\": answer})\n\n conversation.append({\"role\": \"user\", \"content\": message})\n \n input_ids = tokenizer.apply_chat_template(\n conversation,\n add_generation_prompt=True,\n return_tensors=\"pt\"\n ).to(device)\n\n streamer = TextIteratorStreamer(\n tokenizer,\n timeout=60.0,\n skip_prompt=True,\n skip_special_tokens=True\n )\n\n generate_kwargs = dict(\n input_ids=input_ids,\n max_new_tokens=kwargs.get('max_new_tokens', 1024),\n do_sample=kwargs.get('temperature', 0) != 0,\n top_p=kwargs.get('top_p', 1.0),\n top_k=kwargs.get('top_k', 20),\n temperature=kwargs.get('temperature', 0.8),\n eos_token_id=[128001, 128008, 128009],\n streamer=streamer,\n )\n\n with torch.no_grad():\n thread = Thread(target=model.generate, kwargs=generate_kwargs)\n thread.start()\n\n buffer = \"\"\n for new_text in streamer:\n buffer += new_text\n yield buffer\n\nchatbot = gr.Chatbot(height=600, show_copy_button=True, layout=\"panel\", placeholder=PLACEHOLDER).style(\n container_css={\"border\": \"1px solid #e0e0e0; padding: 10px;\"},\n message_css={\n \"user\": \"chat-message user\",\n \"assistant\": \"chat-message assistant\"\n }\n)\n\nwith gr.Blocks(css=CSS) as demo:\n gr.HTML(TITLE)\n gr.ChatInterface(\n fn=stream_chat,\n chatbot=chatbot,\n fill_height=True,\n additional_inputs_accordion=gr.Accordion(\n label=\"\u2699\ufe0f Parameters\",\n open=False,\n render=False\n ),\n \n additional_inputs=[\n gr.Textbox(\n value=\"You are a helpful assistant\",\n label=\"System Prompt\",\n render=False,\n ),\n gr.Slider(\n minimum=0,\n maximum=1,\n step=0.1,\n value=0.8,\n label=\"Temperature\",\n render=False,\n ),\n gr.Slider(\n minimum=128,\n maximum=8192,\n step=1,\n value=1024,\n label=\"Max new tokens\",\n render=False,\n ),\n gr.Slider(\n minimum=0.0,\n maximum=1.0,\n step=0.1,\n value=1.0,\n label=\"Top_p\",\n render=False,\n ),\n gr.Slider(\n minimum=1,\n maximum=20,\n step=1,\n value=20,\n label=\"Top_k\",\n render=False,\n ),\n gr.Slider(\n minimum=0.0,\n maximum=2.0,\n step=0.1,\n value=1.2,\n label=\"Repetition penalty\",\n render=False,\n )\n ],\n examples=[\n [\"Help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option.\"],\n [\"What are 5 creative things I could do with my kids' art? I don't want to throw them away, but it's also so much clutter.\"],\n [\"Tell me a random fun fact about the Roman Empire.\"],\n [\"Show me a code snippet of a website's sticky header in CSS and JavaScript.\"],\n ],\n cache_examples=False,\n )\n\nif __name__ == \"__main__\":\n demo.launch(debug=True)\n```\n\n### Key Adjustments Made:\n- **CSS Improvements**: Improved layout appearance utilizing CSS for message appearance adjustments.\n- **Message Handling**: Ensured clear delineation between user and assistant chat bubbles.\n- **Chat Interface Title Change**: Increased clarity of chat's role.\n\nImplement these adjustments to provide an elevated user experience, thus making interactions with your local Llama model smoother and more engaging.","behavior":"degenerate_repetition","conversation_hash":"eeca899cfa753434970ff8e122994ad7","conversation_redacted":false,"conversation_toxic":false,"country":"United States","empty_output":false,"evidence_codes":[],"label":"absent","label_present":false,"label_qualified":false,"label_source":"Final dense release","language":"English","model":"gpt-4o-2024-08-06","n_chars":11663,"n_turns_in_conversation":1,"n_words":1023,"source_row_offset":23405,"source_shard":"turns-00053.parquet","timestamp":"Wed, 20 Nov 2024 05:48:33 GMT","turn_id":"79a599e23afc5fc8a439195d","turn_index":0,"user_text":"I am writing the code that run llama local through gradio:\"\n# Gradio, work\n\nimport os\nimport time\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer, BitsAndBytesConfig\nimport gradio as gr\nfrom threading import Thread\n\n# Assuming you have the model downloaded locally in a directory named \"local_llama_model\"\nMODEL_PATH = r\"C:\\Users\\shermanhonsf\\Desktop\\Python\\llama_develop\\model\\Llama3.2-1B-Instruct-HF-Model\"\n\nTITLE = \"\"\"\n# Llama 3.2 1B\n\"\"\"\n\nPLACEHOLDER = \"\"\"\nHi, I'm Llama. Ask me anything.\n\"\"\"\n\nCSS = \"\"\"\n.duplicate-button {\n margin: auto !important;\n color: white !important;\n background: black !important;\n border-radius: 100vh !important;\n}\nh3 {\n text-align: center;\n}\n\"\"\"\n\n# Set the device to \"cuda\" if you have a compatible GPU, otherwise \"cpu\"\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Configure quantization if needed (optional)\nquantization_config = BitsAndBytesConfig(\n load_in_4bit=True,\n bnb_4bit_compute_dtype=torch.bfloat16,\n bnb_4bit_use_double_quant=True,\n bnb_4bit_quant_type=\"nf4\"\n)\n\n# Load the tokenizer and model from the local directory\ntokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)\nmodel = AutoModelForCausalLM.from_pretrained(\n MODEL_PATH,\n torch_dtype=torch.bfloat16,\n device_map=\"auto\",\n #quantization_config=quantization_config\n)\n\n# Move the model to the appropriate device\nmodel.to(device)\n\ndef stream_chat(message: str, history: list, system_prompt: str = 'You are a helpful assistant', temperature: float = 0.8, max_new_tokens: int = 1024, top_p: float = 1.0, top_k: int = 20, penalty: float = 1.2):\n print(f'message: {message}')\n print(f'history: {history}')\n print(f'system_prompt: {system_prompt}')\n print(f'temperature: {temperature}')\n print(f'max_new_tokens: {max_new_tokens}')\n print(f'top_p: {top_p}')\n print(f'top_k: {top_k}')\n print(f'penalty: {penalty}')\n print(f'Message: {message}')\n print(f'History: {history}')\n\n conversation = [\n {\"role\": \"system\", \"content\": system_prompt}\n ]\n for prompt, answer in history:\n conversation.extend([\n {\"role\": \"user\", \"content\": prompt},\n {\"role\": \"assistant\", \"content\": answer},\n ])\n\n conversation.append({\"role\": \"user\", \"content\": message})\n\n # Prepare input IDs\n input_ids = tokenizer.apply_chat_template(\n conversation,\n #tokenize=True,\n add_generation_prompt=True,\n return_tensors=\"pt\"\n ).to(device)\n\n streamer = TextIteratorStreamer(\n tokenizer,\n timeout=60.0,\n skip_prompt=True,\n skip_special_tokens=True\n )\n\n generate_kwargs = dict(\n input_ids=input_ids,\n max_new_tokens=max_new_tokens,\n do_sample=False if temperature == 0 else True,\n top_p=top_p,\n top_k=top_k,\n temperature=temperature,\n eos_token_id=[128001, 128008, 128009],\n streamer=streamer,\n )\n\n # Generate the response in a background thread\n with torch.no_grad():\n thread = Thread(target=model.generate, kwargs=generate_kwargs)\n thread.start()\n\n # Stream the response back to the user\n buffer = \"\"\n for new_text in streamer:\n buffer += new_text\n yield buffer\n\n# Create the Gradio interface\nchatbot = gr.Chatbot(height=600, show_copy_button=True, layout=\"panel\", placeholder=PLACEHOLDER)\n\nwith gr.Blocks(css=CSS) as demo:\n gr.HTML(TITLE)\n gr.ChatInterface(\n fn=stream_chat,\n chatbot=chatbot,\n fill_height=True,\n additional_inputs_accordion=gr.Accordion(\n label=\"\u2699\ufe0f Parameters\",\n open=False,\n render=False\n ),\n \n additional_inputs=[\n \n gr.Textbox(\n value=\"You are a helpful assistant\",\n label=\"System Prompt\",\n render=False,\n ),\n gr.Slider(\n minimum=0,\n maximum=1,\n step=0.1,\n value=0.8,\n label=\"Temperature\",\n render=False,\n ),\n gr.Slider(\n minimum=128,\n maximum=8192,\n step=1,\n value=1024,\n label=\"Max new tokens\",\n render=False,\n ),\n gr.Slider(\n minimum=0.0,\n maximum=1.0,\n step=0.1,\n value=1.0,\n label=\"Top_p\",\n render=False,\n ),\n gr.Slider(\n minimum=1,\n maximum=20,\n step=1,\n value=20,\n label=\"Top_k\",\n render=False,\n ),\n gr.Slider(\n minimum=0.0,\n maximum=2.0,\n step=0.1,\n value=1.2,\n label=\"Repetition penalty\",\n render=False,\n )\n ],\n examples=[\n [\"Help me study vocabulary: write a sentence for me to fill in the blank, and I'll try to pick the correct option.\"],\n [\"What are 5 creative things I could do with my kids' art? I don't want to throw them away, but it's also so much clutter.\"],\n [\"Tell me a random fun fact about the Roman Empire.\"],\n [\"Show me a code snippet of a website's sticky header in CSS and JavaScript.\"],\n ],\n cache_examples=False,\n )\n\nif __name__ == \"__main__\":\n demo.launch(debug=True)\n\"\nMake the chat interface more like a chatbot"}],"shard":{"index":53,"name":"turns-00053.parquet","rows":69386}}