Copyright TechTarget - All rights reserved https://cyber.law.harvard.edu/rss/rss.html Techtarget Feed Generator en Sun, 19 Apr 2026 17:28:59 GMT https://www.techtarget.com/searchaws editor@techtarget.com <p>A target function, in <a href="https://www.techtarget.com/searchenterpriseai/definition/machine-learning-ML">machine learning</a>, is a mathematical representation of the ideal mapping between <a href="https://www.techtarget.com/searchenterpriseai/definition/supervised-learning">supervised learning</a> inputs and outputs. Target functions are often written as <i>f(x)</i>, where <i>x</i> represents the input and <i>f(x)</i> represents the correct output.</p> <p>In real-world problems, the target function is not directly observable. To approximate the target function, supervised learning models use labeled training data to learn a hypothesis function, written as <i>h(x)</i>, that closely matches the target function.</p> <p>Once a <a href="https://www.techtarget.com/whatis/definition/machine-learning-algorithm">machine learning algorithm</a> converges on a hypothesis function, it can use the h(x) to make predictions about new, unseen data. This process, known as inference, is used in <a href="https://www.techtarget.com/searchbusinessanalytics/definition/predictive-analytics">predictive analytics</a> to identify patterns, forecast outcomes and support data-driven decision-making in real-world applications.</p> <section class="section main-article-chapter" data-menu-title="How a target function works"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How a target function works</h2> <p>In supervised learning, the <a href="https://www.techtarget.com/searchenterpriseai/feature/Comparing-supervised-vs-unsupervised-learning">training process</a> involves feeding labeled examples -- pairs of inputs and corresponding outputs -- into a machine learning algorithm. During training, the algorithm learns a hypothesis function that maps inputs to outputs by identifying patterns in the data.</p> <p>Machine learning models use optimization techniques, such as <a href="https://www.techtarget.com/searchenterpriseai/definition/gradient-descent">gradient descent</a>, to iteratively adjust the parameters of the model so that h(x) closely approximates f(x) for as many inputs as possible. This process can be described mathematically as <i>y = f(x)</i>, where <i>x</i> is the input, <i>y</i> is the correct output and <i>f(x)</i> is the unknown target function.</p> <p>Data scientists and machine learning engineers approximate the target function to build <a href="https://www.techtarget.com/searchenterpriseai/definition/predictive-modeling">predictive models</a>. When successful, a hypothesis function will minimize <a href="https://www.techtarget.com/whatis/definition/prediction-error">prediction error</a> and enable a predictive model to perform well on both the training data and new, unseen data.</p> </section> <section class="section main-article-chapter" data-menu-title="Target function vs. hypothesis function"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Target function vs. hypothesis function</h2> <p>In supervised learning, the target function represents the true, underlying relationship between inputs and outputs in the real world, and the hypothesis function is what the model actually learns and uses to generate new outputs.</p> <p>Understanding the distinction between these two functions is important because it frames the entire learning process as an approximation task. The concept of approximation is essential for understanding how concepts like <a href="https://www.techtarget.com/whatis/definition/overfitting-in-machine-learning">overfitting</a>, underfitting and generalization influence a model's ability to make accurate predictions on new inputs.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/target_function_vs_hypothesis_function-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/target_function_vs_hypothesis_function-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/target_function_vs_hypothesis_function-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/target_function_vs_hypothesis_function-f.png 1280w" alt="Comparison graphic of target function vs. hypothesis function definitions. " height="182" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>The terms 'target function' and 'hypothesis function' are often used interchangeably, but there's an important distinction. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Examples of target functions in machine learning"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Examples of target functions in machine learning</h2> <p>The nature of a target function depends on the type of machine learning problem that's being solved, and whether the solution involves regression, binary classification, multiclass classification or more complex nonlinear mappings.</p> <ul class="default-list"> <li><b>Linear regression.</b> The target function assumes there is a linear relationship between inputs and outputs, e.g., predicting house prices.</li> <li><b>Logistic regression.</b> The target function defines the relationship between inputs and binary class labels, e.g., spam vs. not spam.</li> <li><b>Decision trees.</b> The target function is modeled as a <a href="https://www.techtarget.com/searchenterpriseai/definition/decision-tree-in-machine-learning">nonlinear, hierarchical structure</a> that maps inputs to outputs using a series of rule-based decisions.</li> <li><b>Support vector machines.</b> The model learns a decision boundary -- or a hyperplane -- that separates classes in the feature space to approximate the target function in classification tasks.</li> <li><b>Neural networks.</b> The model approximates complex, often nonlinear target functions using multiple layers of interconnected nodes with activation functions. Activation functions enable the model to learn complex mappings.</li> </ul> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/yOf2ssqJFFk?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Importance of the target function in machine learning"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Importance of the target function in machine learning</h2> <p>In supervised learning, the concept of a target function is important because it guides the learning process and helps define what the model is ultimately trying to approximate. As <a href="https://www.techtarget.com/searchenterpriseai/definition/deep-learning-deep-neural-network">deep learning</a>, <a href="https://www.techtarget.com/searchenterpriseai/definition/reinforcement-learning">reinforcement learning</a> and <a href="https://www.techtarget.com/searchenterpriseai/definition/generative-AI">generative AI</a> models become more advanced, accurately approximating the target function has become essential for building reliable predictive models.</p> <p>Modern systems use a combination of approaches to effectively approximate the target function. They include the following:</p> <ul class="default-list"> <li>Loss functions that quantify <a target="_blank" href="https://www.ibm.com/think/topics/loss-function" rel="noopener">how closely</a> the model's hypothesis function aligns with the target function.</li> <li><a href="https://www.techtarget.com/searchenterpriseai/feature/Machine-learning-regularization-explained-with-examples">Regularization techniques</a> that help prevent overfitting while still capturing the underlying target function.</li> <li><a href="https://www.techtarget.com/searchenterpriseai/feature/Interpretability-vs-explainability-in-AI-and-machine-learning">Explainability tools</a>, such as SHAP and LIME, analyze model behavior and provide insights into how a hypothesis function relates to the predicted outputs.</li> </ul> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/interpretability_vs_explainability-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/interpretability_vs_explainability-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/interpretability_vs_explainability-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/interpretability_vs_explainability-f.png 1280w" alt="Graphic that explains the differences between interpretability vs. explainability." height="319" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Interpretability is about understanding how a model works internally, and explainability is about understanding why it makes specific predictions. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>When a model's hypothesis function approximates the true target function, it can reduce error and improve generalization to unseen data. In turn, this enables machine learning engineers to build AI systems that are more trustworthy, interpretable and aligned with real-world outcomes.</p> <p>Today, improving the approximation of target functions is vital for AI model accuracy, fairness and robustness, especially in high-stakes fields like healthcare, finance and autonomous systems. In these domains, even small prediction errors can lead to significant consequences, such as incorrect diagnoses, <a href="https://www.techtarget.com/searchenterpriseai/definition/machine-learning-bias-algorithm-bias-or-AI-bias">biased</a> lending decisions or unsafe navigation.</p> </section> <section class="section main-article-chapter" data-menu-title="Related concepts to target function"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Related concepts to target function</h2> <p>To fully understand target functions, it helps to be familiar with related machine learning terms:</p> <ul class="default-list"> <li><b>Training data.</b> Examples with known inputs and outputs used to train the model.</li> <li><b>Generalization.</b> The ability of a model to apply the learned function to new, unseen data.</li> <li><b>Overfitting.</b> When a model learns the training data too well, its outputs can be based on noise instead of the underlying patterns it should use to generate new outputs.</li> <li><b>Model capacity.</b> A model's ability to approximate complex target functions; higher-capacity models can learn more complex mappings, but risk overfitting.</li> </ul> <p><em>Learn more about the <a href="https://www.techtarget.com/whatis/feature/History-and-evolution-of-machine-learning-A-timeline">history and evolution of machine learning</a> in this comprehensive essential guide.</em></p> </section> A target function, in machine learning, is a mathematical representation of the ideal mapping between supervised learning inputs and outputs. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/whatis/definition/target-function Tue, 03 Jun 2025 10:00:00 GMT <p>LangChain is an open source framework that enables software developers working with artificial intelligence (<a href="https://www.techtarget.com/searchenterpriseai/definition/AI-Artificial-Intelligence">AI</a>) and its machine learning subset to combine large language models with other external components to develop <a href="https://www.techtarget.com/whatis/definition/large-language-model-LLM">LLM</a>-powered applications.</p> <p>LangChain aims to link powerful LLMs, such as OpenAI's GPT-3.5 and <a href="https://www.techtarget.com/whatis/definition/GPT-4">GPT-4</a>, to an array of external data sources to create and reap the benefits of natural language processing (<a href="https://www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP">NLP</a>) applications. It's also used for creating interfaces that produce humanlike responses and answer questions.</p> <p>Developers, software engineers and data scientists with experience in Python, JavaScript or TypeScript programming languages can use LangChain packages offered in those languages.</p> <p>LangChain was released in 2022 as an open source project by co-founders Harrison Chase and Ankush Gola.</p> <section class="section main-article-chapter" data-menu-title="Why is LangChain important?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Why is LangChain important?</h2> <p>LangChain is a framework that simplifies the process of creating <a href="https://www.techtarget.com/searchenterpriseai/definition/generative-AI">generative AI</a> application interfaces. Developers working on these types of interfaces use various tools to create advanced NLP apps; LangChain streamlines this process. For example, LLMs must access large volumes of <a href="https://www.techtarget.com/searchdatamanagement/definition/big-data">big data</a>, so LangChain organizes these large quantities of data so that they can be accessed with ease.</p> <p>In addition, GPT (<a href="https://www.techtarget.com/searchenterpriseai/feature/ChatGPT-vs-GPT-How-are-they-different">generative pre-trained transformer</a>) models are trained on data available until a specific point in time, known as their knowledge cutoff date. While models are often updated to extend their cutoff date, LangChain can connect AI models directly to data sources to give them knowledge of recent data without limitations.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/zKndCikg3R0?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="What are the benefits of LangChain?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What are the benefits of LangChain?</h2> <p>LangChain offers various benefits, including the following:</p> <ul class="default-list"> <li><b>Open source and community benefits.</b> As an open source framework, LangChain thrives on community contributions and collaboration, and is readily accessible on platforms such as <a href="https://www.techtarget.com/searchitoperations/definition/GitHub">GitHub</a>. Developers can drive innovation by accessing a wealth of resources, tutorials, documentation and support from fellow LangChain users.</li> <li><b>Modular design.</b> LangChain's modular architecture enables developers to mix, match and customize components for specific needs.</li> <li><b>Simplified development.</b> LangChain offers a standardized interface that enables developers to easily switch between different LLMs, streamline workflows and reduce integration complexity. For example, they can switch between LLMs from providers such as OpenAI or <a href="https://www.techtarget.com/whatis/definition/Hugging-Face">Hugging Face</a> with minimal code changes.</li> <li><b>Repurposed LLMs.</b> LangChain lets organizations repurpose LLMs for domain-specific applications without retraining, enabling development teams to enhance model responses with proprietary information, such as summarizing internal documents. The retrieval-augmented generation (<a href="https://www.techtarget.com/searchenterpriseai/definition/retrieval-augmented-generation">RAG</a>) workflow further improves response accuracy and reduces model <a href="https://www.techtarget.com/searchenterpriseai/tip/A-short-guide-to-managing-generative-AI-hallucinations">hallucination of generative AI</a> by introducing relevant information during prompting.</li> <li><b>Interactive applications.</b> LangChain enables interactive applications through real-time communication with language models. For example, its modular components can be used to create interactive applications such as chatbots and AI assistants that engage users in real time.</li> </ul> </section> <section class="section main-article-chapter" data-menu-title="What are the features of LangChain?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What are the features of LangChain?</h2> <p>LangChain is made up of the following modules that ensure the multiple components needed to make an effective NLP app can run smoothly:</p> <ul class="default-list"> <li><b>Model interaction.</b> Also called Model I/O, this module enables LangChain to interact with any language model and perform tasks such as managing inputs to the model and extracting information from its outputs.</li> <li><b>Prompt templates.</b> LangChain includes prompt template modules that enable developers to create structured prompts for LLMs. These templates can incorporate examples and specify output formats, facilitating smoother interactions and more accurate responses from the models.</li> <li><b>Data connection and retrieval.</b> Data that LLMs access can be transformed, stored in databases and retrieved from those databases through queries with this module.</li> <li><b>Chains.</b> Using LangChain to build more complex apps might require other components or even more than one LLM. This module links multiple LLMs with other components or LLMs. This is referred to as an LLM chain.</li> <li><b>Agents.</b> The agent module lets LLMs decide the best steps or actions to take to solve problems. It orchestrates a series of complex commands to LLMs and other tools to get them to respond to specific requests.</li> <li><b>Memory.</b> The memory module helps an LLM remember the context of its interactions with users. Depending on the specific use, both short-term and long-term memory can be added to a model.</li> <li><b>Retrieval modules.</b> LangChain supports the development of RAG systems with tools for transforming, storing and retrieving information to enhance language model responses. This enables developers to produce <a href="https://www.techtarget.com/searchenterpriseai/tip/Embedding-models-for-semantic-search-A-guide">semantic representations with word embeddings</a> and store them in local or cloud-based <a href="https://www.techtarget.com/searchdatamanagement/definition/What-is-a-vector-database">vector databases</a>.</li> </ul> </section> <section class="section main-article-chapter" data-menu-title="What are LangChain integrations?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What are LangChain integrations?</h2> <p>LangChain typically builds applications using integrations with LLM providers and external sources where data can be found and stored. For example, LangChain can build <a href="https://www.techtarget.com/searchcustomerexperience/infographic/The-evolution-of-chatbots-and-generative-AI">chatbots</a> or question-answering systems by integrating an LLM with data sources or stores, including relational or graph databases, text files, knowledge bases or unstructured data. This enables an app to take user-input text, process it and retrieve the best answers from any of these sources. In this sense, LangChain integrations use the most up-to-date <a href="https://www.theserverside.com/tutorial/Natural-language-programming-using-GPTScript">NLP technology to build effective apps</a>.</p> <p>Other potential integrations include cloud storage platforms, such as <a href="https://www.techtarget.com/searchaws/definition/Amazon-Web-Services">Amazon Web Services</a>, <a href="https://www.techtarget.com/searchcloudcomputing/definition/Google-Cloud-Platform">Google Cloud</a> and <a href="https://www.techtarget.com/searchcloudcomputing/definition/Windows-Azure">Microsoft Azure</a>, as well as vector databases. A vector database can store large volumes of high-dimensional data -- such as videos, images and long-form text -- as mathematical representations that make it easier for an application to query and search for those data elements.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/a_langchain_app_in_its_most_basic_form-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/a_langchain_app_in_its_most_basic_form-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/a_langchain_app_in_its_most_basic_form-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/a_langchain_app_in_its_most_basic_form-f.png 1280w" alt="A diagram showing how a LangChain app works." height="241" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>LangChain augments existing LLMs so that they can handle user queries and prompts. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="How to create prompts in LangChain"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How to create prompts in LangChain</h2> <p><a href="https://www.techtarget.com/searchenterpriseai/definition/AI-prompt">Prompts</a> serve as inputs to the LLM that instruct it to return a response, which is often an answer to a query. This response is also referred to as an output. A prompt must be designed and executed correctly to increase the likelihood of a well-written and accurate response from a language model. That's why <a href="https://www.techtarget.com/whatis/feature/Skills-needed-to-become-a-prompt-engineer">prompt engineering is an emerging science</a> that has received more attention in recent years.</p> <p>Prompts can be generated easily in LangChain executions using a prompt template, which serves as instructions for the underlying LLM. Prompt templates can vary in specificity. They can be designed to pose simple questions to a language model. They can also be used to provide a set of explicit instructions to a language model with enough detail and examples to retrieve a high-quality response.</p> <p>With Python programming, LangChain has a premade prompt template that takes the form of structured text. The following steps are required to use this:</p> <ol class="default-list"> <li><b>Install Python.</b> A recent version of Python must be installed. Once the Python <a href="https://www.techtarget.com/searchdatacenter/definition/shell-script">shell</a> terminal is open, enter the following command to install just the bare minimum requirements of LangChain for the sake of this example:</li> </ol> <pre class="language-python"><code>pip install langchain</code></pre> <ol start="2" class="default-list"> <li><b>Add integrations.</b> LangChain typically requires at least one integration. OpenAI is a prime example. To use OpenAI's LLM application programming interfaces, a developer must create an account on the OpenAI website and retrieve the <a href="https://www.techtarget.com/searchapparchitecture/definition/application-program-interface-API">API</a> <a href="https://www.theserverside.com/video/How-to-get-an-OpenAI-API-key-for-ChatGPT">access key</a>. Then, using the following code snippet, install OpenAI's Python package and enter the key for access to the APIs:</li> </ol> <pre class="language-python"><code>pip install openai from langchain.llms import OpenAI llm = OpenAI(openai_api_key="...")</code></pre> <ol start="3" class="default-list"> <li><b>Import the prompt template.</b> Once these basic steps are complete, LangChain's prompt template method must be imported. The code snippet shown below does this:</li> </ol> <pre class="language-python"><code>from langchain import PromptTemplate prompt_template = PromptTemplate.from_template( "Tell me an {adjective} fact about {content}." ) prompt_template.format(adjective="interesting", content="zebras") "Tell me an interesting fact about zebras."</code></pre> <p>In this scenario, the language model would be expected to take the two input variables -- the adjective and the content -- and produce a fascinating fact about zebras as its output.</p> </section> <section class="section main-article-chapter" data-menu-title="How to develop applications in LangChain"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How to develop applications in LangChain</h2> <p>LangChain is built to develop apps powered by language model functionality. There are different ways to do this, but the process typically entails the following key steps:</p> <ol class="default-list"> <li><b>Set up the environment.</b> The first step is to install LangChain with all its necessary dependencies. In addition, developers should ensure they have access to an LLM API, such as OpenAI, and obtain the API key for integration.</li> <li><b>Define the application.</b> An application developer must first define a specific use case for the application. This also means determining its scope, including requirements such as any needed integrations, components and LLMs.</li> <li><b>Build functionality.</b> Developers use prompts to build the functionality or logic of the intended app.</li> <li><b>Customize functionality.</b> LangChain lets developers modify their code to create customized functionality that meets the needs of the use case and shapes the application's behavior.</li> <li><b>Fine-tune LLMs.</b> It's important to choose the appropriate LLM for the job and to fine-tune it to adhere to the needs of the use case.</li> <li><b>Cleanse data.</b> Using <a href="https://www.techtarget.com/searchdatamanagement/definition/data-scrubbing">data cleansing</a> techniques ensures clean and accurate data sets. Security measures should also be executed to protect sensitive data.</li> <li><b>Test.</b> Regularly testing LangChain apps ensures they continue to run smoothly.</li> </ol> </section> <section class="section main-article-chapter" data-menu-title="LangChain vs. LangSmith"> <h2 class="section-title"><i class="icon" data-icon="1"></i>LangChain vs. LangSmith</h2> <p>LangChain and LangSmith play specific roles in the landscape of LLM applications. The LangChain framework helps developers create LLM-powered applications by offering tools that build intricate workflows and integrate different components.</p> <p>The LangSmith platform is focused on monitoring, testing, debugging and evaluating these applications in production to ensure they operate reliably and efficiently.</p> <p>In essence, LangChain aids in the creation of applications, while LangSmith enhances their operational management and <a href="https://www.techtarget.com/searchsoftwarequality/definition/quality-assurance">quality assurance</a>.</p> </section> <section class="section main-article-chapter" data-menu-title="LangChain vs. LangGraph"> <h2 class="section-title"><i class="icon" data-icon="1"></i>LangChain vs. LangGraph</h2> <p>Similar to LangChain and LangSmith, LangGraph is also a framework for LLMs. LangChain offers a standard interface for building simple applications and workflows using linear chains and retrieval flows.</p> <p>In contrast, LangGraph builds on LangChain by enabling cyclical graphs, which facilitate the development of complex, stateful and multi-actor applications. This feature enables more control over interactions, including conditional paths and opportunities for human intervention, making LangGraph suitable for advanced agent-based software development.</p> </section> <section class="section main-article-chapter" data-menu-title="Examples and use cases for LangChain"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Examples and use cases for LangChain</h2> <p>The LLM-based applications LangChain is capable of building can be applied to multiple advanced use cases within various industries and vertical markets, such as the following:</p> <ul class="default-list"> <li><b>Customer service chatbots.</b> LangChain enables advanced chat applications to handle complex questions and user transactions. These applications can understand and maintain a user's context throughout a conversation in the same way as <a href="https://www.techtarget.com/whatis/definition/ChatGPT">ChatGPT</a>. AI is widely used to <a href="https://www.techtarget.com/searchcustomerexperience/post/How-AI-can-benefit-CX-and-customer-service">enhance customer experience and service</a>.</li> <li><b>Coding assistants.</b> It's possible to build coding assistants with the help of LangChain. Using LangChain and OpenAI's API, developers can create a tool to assist those in the tech sector with enhancing their coding skills and improving productivity.</li> <li><b>Healthcare.</b> AI has entered healthcare in several ways. LLM-centric LangChain applications help doctors make diagnoses. They also automate rote, repetitive administrative tasks, such as scheduling patient appointments, enabling healthcare workers to focus on more important work.</li> <li><b>Marketing and e-commerce.</b> Businesses use e-commerce platforms with LLM functionality to better engage customers and expand their customer base. An application that can understand consumer purchasing patterns and product descriptions can generate product recommendations and compelling descriptions for potential customers.</li> <li><b>Summarization tasks.</b> LangChain can help condense large volumes of text into concise summaries. This is particularly useful in fields such as journalism, research and content creation, where vital information must be accessed quickly.</li> <li><b>Data augmentation.</b> LangChain can enhance data by generating new data that resembles the existing data. This includes creating variations of the data by paraphrasing or slightly altering its context. Data augmentation is useful for training machine learning models and developing new data sets.</li> <li><b>Media adaptation.</b> LangChain can help the entertainment industry by enabling dubbing and subtitling of content, making movies and TV shows accessible to a broader audience. This improves the viewer experience and expands the content's international reach.</li> </ul> <p><em>Discover how advancements in large language models and generative AI are revolutionizing industries and driving innovation. Explore future <a href="https://www.techtarget.com/searchbusinessanalytics/news/366608491/GenAI-evolving-remains-dominant-data-and-analytics-trend">trends that are shaping AI</a>.</em></p> </section> LangChain is an open source framework that enables software developers working with artificial intelligence (AI) and its machine learning subset to combine large language models with other external components to develop LLM-powered applications. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchenterpriseai/definition/LangChain Thu, 02 Jan 2025 17:00:00 GMT <p>Robotics is a branch of engineering and computer science that involves the conception, design, manufacture and operation of <a href="https://www.techtarget.com/searchenterpriseai/definition/robot">robots</a>. The objective of the robotics field is to create intelligent machines that can assist humans in a variety of ways.</p> <p>Robotics can take on a number of forms. A robot might resemble a human or be in the form of a robotic application, such as <a href="https://www.techtarget.com/searchcio/definition/RPA">robotic process automation</a>, which simulates how humans engage with software to perform repetitive, rules-based tasks.</p> <p>While the field of robotics and exploration of the potential uses and functionality of robots have grown substantially in the 21st century, the idea certainly isn't new.</p> <section class="section main-article-chapter" data-menu-title="The early history of robotics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>The early history of robotics</h2> <p>The term <i>robotics</i> is an extension of the word <i>robot</i>. One of its first uses came from Czech writer Karel Čapek, who used the word in his 1920 play, <i>Rossum's Universal Robots</i>.</p> <p>However, in the 1940s, the Oxford English Dictionary credited science fiction author Isaac Asimov for being the first person to use the term. In Asimov's story, he suggested three principles to guide the behavior of autonomous robots and smart machines:</p> <ol class="default-list"> <li>Robots must never harm human beings.</li> <li>Robots must follow instructions from humans without violating rule 1.</li> <li>Robots must protect themselves without violating the other rules.</li> </ol> <p>His three laws of robotics have survived to the present day. However, it wasn't until a couple of decades later, in 1961, that the first programmable robot -- called Unimate, derived from <i>universal automation</i> -- was created based on designs from the '50s to move scalding metal pieces from a die-cast machine. The Stanford Research Institute's robot dubbed Shakey followed suit in 1966 as the first <a href="https://www.techtarget.com/iotagenda/definition/mobile-robot-mobile-robotics">mobile robot</a>, thanks to software and hardware that enabled it to sense and grasp the environment, though in a limited capacity.</p> </section> <section class="section main-article-chapter" data-menu-title="Robotics applications"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Robotics applications</h2> <p>Today, industrial robots, as well as many other types of robots, are used to perform repetitive tasks. They can take the form of a robotic arm, a <a href="https://www.techtarget.com/whatis/video/An-explanation-of-collaborative-robots">collaborative robot</a> (cobot), a robotic exoskeleton or traditional humanoid robots.</p> <p>Industrial robots and robot arms are used by manufacturers and warehouses, such as those owned by <a href="https://www.techtarget.com/whatis/definition/Amazon">Amazon</a> and Best Buy.</p> <p>To function, a combination of computer programming and <a href="https://www.techtarget.com/whatis/definition/algorithm">algorithms</a>, a remotely controlled manipulator, actuators, <a href="https://www.techtarget.com/whatis/definition/control-system">control systems</a> -- action, processing and perception -- real-time sensors and an element of automation help to inform what a robot or robotic system does.</p> <p>Some additional applications for robotics include the following:</p> <ul class="default-list"> <li><b>Home electronics.</b> Vacuum cleaners and lawnmowers can be programmed to automatically perform tasks without human intervention.</li> <li><b>Home monitoring.</b> This includes specific types of robots that can monitor home energy usage or provide home security monitoring services, such as Amazon Astro.</li> <li><b>Artificial intelligence (AI).</b> Robotics is widely used in <a href="https://www.techtarget.com/searchenterpriseai/definition/AI-Artificial-Intelligence">AI</a> and <a href="https://www.techtarget.com/searchenterpriseai/definition/machine-learning-ML">machine learning</a> (ML) processes, specifically for object recognition, <a href="https://www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP">natural language processing</a>, predictive maintenance and process automation.</li> <li><b>Data science.</b> The field of <a href="https://www.techtarget.com/searchenterpriseai/definition/data-science">data science</a> relies on robotics to perform tasks including data cleaning, data automation, data analytics and anomaly detection.</li> <li><b>Law enforcement and military.</b> Both law enforcement and the military rely heavily on robotics, as it can be used for surveillance and reconnaissance missions. Robotics is also used to improve soldier mobility on the battlefield.</li> <li><b>Mechanical engineering.</b> Robotics is widely used in manufacturing operations, such as the inspection of pipelines for corrosion and testing the structural integrity of buildings.</li> <li><b>Mechatronics.</b> Robotics aids in the development of <a href="https://www.techtarget.com/searcherp/definition/smart-factory">smart factories</a>, robotics-assisted surgery devices and <a href="https://www.techtarget.com/searchenterpriseai/feature/Autonomous-vehicle-technology-advancing-amid-big-challenges">autonomous vehicles</a>.</li> <li><b>Nanotechnology.</b> Robotics is extensively used in the manufacturing of <a href="https://www.techtarget.com/iotagenda/definition/micro-electromechanical-systems-MEMS">microelectromechanical systems</a>, which is a process used to create tiny integrated systems.</li> <li><b>Bioengineering and healthcare.</b> <a href="https://www.techtarget.com/searchenterpriseai/definition/robotic-surgery-robot-assisted-surgery">Surgical robots</a>, assistive robots, lab robots and telemedicine robots are all examples of robotics used in the fields of healthcare and bioengineering.</li> <li><b>Aerospace.</b> Robotics can be used for drilling, painting, coating, inspection and maintenance of aircraft components.</li> </ul> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/Kf9rgbq-G2k?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Types of robotics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Types of robotics</h2> <p>Robots are designed to perform specific tasks and operate in different environments. The following are some common types of robots used across various industries:</p> <ul class="default-list"> <li><b>Industrial robots.</b> Frequently used in manufacturing and warehouse settings, these large <a href="https://www.techtarget.com/searchenterpriseai/feature/How-warehouse-automation-robotics-transformed-the-supply-chain">programmable robots are transforming the supply chain</a> by performing tasks such as welding, painting, assembling and material handling.</li> <li><b>Service robots.</b> These robots are used in a variety of fields in different scenarios, such as domestic chores, hospitality, retail and healthcare. Examples include cleaning robots, entertainment robots and personal assistance robots.</li> <li><b>Medical robots.</b> These robots help with surgical procedures, rehabilitation and diagnostics in healthcare settings. Robotic surgery systems, exoskeletons and artificial limbs are a few examples of medical robots.</li> <li><b>Autonomous vehicles.</b> These robots are mainly used for transportation purposes and can include self-driving cars, drones and autonomous delivery robots. They navigate and make decisions using advanced sensors and <a href="https://www.techtarget.com/searchenterpriseai/tip/Types-of-AI-algorithms-and-how-they-work">AI algorithms</a>.</li> <li><b>Humanoid robots.</b> These robots are programmed to imitate and mimic human movements and actions. They look humanlike and are employed in research, entertainment and human-robot interactions.</li> <li><b>Cobots.</b> Contrary to the majority of other types of robots, which do their tasks alone or in entirely separated work environments, <a href="https://www.techtarget.com/whatis/definition/collaborative-robot-cobot">cobots</a> can share workplaces with human employees, enabling them to work more productively. They're typically used to remove costly, dangerous or time-consuming tasks from routine workflows. Cobots can occasionally recognize and respond to human movement.</li> <li><b>Agricultural robots.</b> These robots are used in farming and agricultural applications. They can plant, harvest, apply pesticides and check crop health.</li> <li><b>Exploration and space robots.</b> These robots are used in missions to explore space as well as in harsh regions on Earth. Examples include underwater exploration robots and rovers used on Mars expeditions.</li> <li><b>Defense and military robots.</b> These robots aid military tasks and operations including surveillance, bomb disposal and search-and-rescue missions. They're specifically designed to operate in unknown terrains.</li> <li><b>Educational robots.</b> These robots are created to instruct and educate kids about robotics, programming and problem-solving. Kits and platforms for hands-on learning in academia are frequent examples of educational robots.</li> <li><b>Entertainment robots.</b> Created for entertainment purposes, these robots come in the form of robotic pets, humanoid companions and interactive toys.</li> </ul> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/types_of_robots-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/types_of_robots-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/types_of_robots-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/types_of_robots-f.png 1280w" alt="Types of robots." height="226" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>There are many different kinds of robots suited to a variety of tasks. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Machine learning in robotics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Machine learning in robotics</h2> <p>Machine learning and robotics intersect in a field known as robot learning. Robot learning is the study of techniques that enable a robot to acquire new knowledge or skills through ML algorithms.</p> <p>Some applications that have been explored by robot learning include grasping objects, object categorization and even linguistic interaction with a human peer. Learning can happen through self-exploration or guidance from a human operator.</p> <p>To learn, intelligent robots must accumulate facts through human input or sensors. Then, the robot's processing unit compares the newly acquired data with previously stored information to predict the best course of action based on the data it has acquired. However, it's important to understand that a robot can only solve problems that it's built to solve. It doesn't have general analytical abilities.</p> <p>Some examples of current applications of machine learning in robotics include the following:</p> <ul class="default-list"> <li><b>Computer vision.</b> Robots can perceive, identify and navigate their environments with the help of machine vision, which uses ML algorithms and sensors. <a href="https://www.techtarget.com/searchenterpriseai/ehandbook/Computer-vision-AI-looks-beyond-the-narrow-into-the-mainstream">Computer vision is used in a wide range of settings</a>, including manufacturing procedures, such as material inspection and pattern and signature recognition.</li> <li><b>Self-supervised learning.</b> By using large sets of data, robots can be taught to perform tasks without being specifically trained to do so, such as in <a href="https://www.techtarget.com/searchenterpriseai/definition/neural-network">neural networks</a>. Self-supervised learning can be used to increase the ability of robots to adapt to changing environments.</li> <li><b>Imitation learning.</b> This entails educating robots to replicate human behavior by demonstrating desirable actions to them. This can be used to improve the speed and accuracy of automated procedures.</li> <li><b>Assistive robotics.</b> Machine learning can be used to create robotic devices that help people with daily tasks such as mobility and household duties. For example, wheelchair-mounted robot arms can offer greater independence to people with limited mobility in their upper extremities.</li> <li><b>Reinforcement learning.</b> This entails teaching robots how to carry out challenging tasks through the use of trial-and-error techniques to make robotic systems more effective and efficient.</li> </ul> </section> <section class="section main-article-chapter" data-menu-title="The pros and cons of robotics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>The pros and cons of robotics</h2> <p>Robotic systems are coveted in many industries because they can increase accuracy, reduce costs and increase safety for human beings.</p> <p>Common advantages of robotics include the following:</p> <ul class="default-list"> <li><b>Safety.</b> Safety is arguably one of robotics' greatest benefits, as many dangerous or unhealthy environments no longer require the human element. Examples include the nuclear industry, space, defense and maintenance. With robots or robotic systems, workers can avoid exposure to hazardous chemicals and even limit psychosocial and ergonomic health risks.</li> <li><b>Increased productivity.</b> Robots don't readily become tired or worn out as humans do. They can work continuously without breaks while performing repetitive jobs, which boosts productivity.</li> <li><b>Accuracy.</b> Robots can perform precise tasks with greater consistency and accuracy than humans can. This eliminates the risk of errors and inconsistencies.</li> <li><b>Flexibility.</b> Robots can be programmed to carry out a variety of tasks and are easily adaptable to new use cases.</li> <li><b>Cost savings.</b> By automating repetitive tasks, robots can reduce labor costs.</li> </ul> <p>However, despite these benefits, robotics also comes with the following drawbacks:</p> <ul class="default-list"> <li><b>Task suitability.</b> Certain tasks are simply better suited for humans -- for example, those jobs that require creativity, adaptability and critical decision-making skills.</li> <li><b>Economic problems.</b> Since robots can perform most jobs that humans do with more precision, speed and accuracy, there's always a potential risk that they could eventually replace human jobs.</li> <li><b>Cost.</b> Most robotic systems have a high initial cost. It can also cost a lot to repair and maintain robots.</li> <li><b>Increased dependency.</b> Overreliance on robots can result in a decrease in human talents and problem-solving abilities as well as an increase in technological dependence.</li> <li><b>Security risks.</b> There's always a risk of robotic devices getting hacked or hijacked, especially if they're being used for defense and security purposes.</li> <li><b>Power requirements.</b> Robots consume a lot of energy and constant power to operate. Regular upkeep and maintenance are also needed to keep them in good working condition.</li> </ul> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/what_can_robots_do-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/what_can_robots_do-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/what_can_robots_do-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/what_can_robots_do-f.png 1280w" alt="Robotics pros and cons." height="269" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Although robots can perform certain tasks better than humans, they also have their shortcomings. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="The future of robotics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>The future of robotics</h2> <p>Robots are becoming more efficient, flexible and autonomous as AI, machine learning and sensor technologies progress. They're predicted to play an increasingly essential role in a variety of industries and applications in the future. A study by Spherical Insights and Consulting estimated that the size of the global robotics market will increase from $25.82 billion in 2022 to $115.88 billion by 2032.</p> <p>However, there are also possible concerns about employment losses associated with the exponential adoption of robotics across various industries. According to Oxford Economics, up to 20 million manufacturing jobs could be <a href="https://resources.oxfordeconomics.com/how-robots-change-the-world" target="_blank" rel="noopener">lost to robots</a> by 2030. On the flip side, robots are likely to generate new professional opportunities in fields such as programming and maintenance, despite the looming possibility of job losses.</p> <p><em>Manufacturing organizations are using robotics and AI to improve operations and boost productivity. Explore the 10 <a href="https://www.techtarget.com/searcherp/feature/10-AI-use-cases-in-manufacturing">use cases of robotics in the manufacturing industry</a>.</em></p> </section> Robotics is a branch of engineering that involves the conception, design, manufacture and operation of robots. https://cdn.ttgtmedia.com/visuals/digdeeper/1.jpg https://www.techtarget.com/whatis/definition/robotics Wed, 04 Sep 2024 13:15:00 GMT <p>AWS Migration Hub is a service from Amazon Web Services that enables AWS users to simplify migration to the AWS public cloud. Its purpose is to simplify discovery, assessment, planning and execution to streamline organizations' end-to-end migrations and modernization journeys.</p> <p>AWS Migration Hub supports and integrates with native migration tools, such as AWS Server Migration Service and AWS Database Migration Service, as well as several partner tools to provide accurate <a href="https://www.techtarget.com/searchstorage/definition/data-migration">data migrations</a>.</p> <section class="section main-article-chapter" data-menu-title="Purpose of AWS Migration Hub"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Purpose of AWS Migration Hub</h2> <p>With AWS Migration Hub, business users can plan migrations, track the migration status of each application, get full visibility into the entire application portfolio, and collect and view data from on-premises resources including servers. AWS Migration Hub facilitates discovery of applications, migration execution, grouping servers into applications and accelerating application <a href="https://www.techtarget.com/searchapparchitecture/definition/refactoring">refactoring</a>. The service also enables users to manage existing applications and <a href="https://www.techtarget.com/searchapparchitecture/definition/microservices">microservices</a> as one application.</p> <p>AWS Migration Hub provides guidance, tools, automation recommendations and prescriptive plans to accelerate <a href="https://www.techtarget.com/searchcloudcomputing/definition/cloud-migration">cloud migration</a>. It also provides proven journey templates and facilitates cross-team collaboration and tracking, enabling businesses to define the migration and modernization strategy that is most likely to help accelerate the process of rehosting, <a href="https://www.techtarget.com/searchcloudcomputing/definition/replatforming">replatforming</a> or refactoring applications. The AWS Migration Hub console enables users to track the migration progress of their applications into AWS.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/-xanQ3aUWms?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Important features of AWS Migration Hub"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Important features of AWS Migration Hub</h2> <p>AWS Migration Hub includes numerous features to make it easy for organizations to plan, execute and track migrations. For example, it provides journey templates to offer guidance on the recommended tasks for all migration phases, customizable step-by-step <a href="https://www.techtarget.com/searchnetworking/definition/run-book">runbooks</a> to execute various subtasks, and predefined workflow templates to accelerate migration and enhance user experiences. By leveraging the guidance, <a href="https://www.techtarget.com/whatis/definition/template">templates</a> and runbooks, businesses can speed up project planning and reduce dependency on outside cloud consultants.</p> <p>Users can import information about on-premises servers and applications, and discover applications to determine which ones can be migrated and modernized. Another option is to use discovery tools like AWS Discovery Agent or the agentless AWS Discovery Collector for VMware environments if a deeper discovery process is required.</p> <p>A network visualization feature enables users to quickly identify servers and associated dependencies, identify server roles, and group servers into applications. Doing so can help to accelerate migration planning. To build a migration and modernization strategy for on-premises or AWS applications, AWS Migration Hub provides Strategy Recommendations, a feature that analyzes applications and determines the optimal migration strategies.</p> <p>Once the migration begins, users can use the AWS Migration Hub <a href="https://www.techtarget.com/searchcio/definition/executive-dashboard">dashboard</a> to view the migration status and metrics. The dashboard is particularly useful to track the progress of rehost and replatform migrations, and to identify and troubleshoot issues.</p> <p>For incremental application refactoring to microservices, AWS Migration Hub provides a chargeable Refactor Spaces feature that removes the undifferentiated work of building and operating AWS infrastructure, minimizes the risk of evolving applications into microservices, and bridges networking across AWS accounts. Moreover, Refactor Spaces maintains the transparency of the underlying architecture changes by providing an application proxy.</p> </section> <section class="section main-article-chapter" data-menu-title="Start a migration with AWS Migration Hub"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Start a migration with AWS Migration Hub</h2> <p>To help users get started with a migration using AWS Migration Hub, the tool prompts them to choose a home region from the <a href="https://www.techtarget.com/searchaws/definition/AWS-Management-Console">AWS Management Console</a>. The home region is where the migration tracking data will be stored. That said, an AWS login is not required to start the AWS migration; applications can also be migrated outside the AWS Console.</p> <p>Whether or not a login is used, the AWS Migration Hub prompts users to provide information about their existing environment and to specify their migration goals. After doing this, they can either select a built-in journey template or customize it to define their end-to-end migration project.</p> <p>Prior to migration, users can perform discovery on local resources using a discovery tool. The tools -- Agentless Collector, Discovery Agent and Migration Evaluator -- are available on the Discovery Tools page of the AWS Migration Hub console navigation pane. After choosing a discovery tool, users can view detailed information about discovered servers and explore server network connections.</p> <p>It's important to authorize the chosen migration tool to communicate the migration status to Migration Hub in the home region. Without this authorization, AWS Migration Hub will not track the migration.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/editorial/sAWS_20180510_MigrationHub.png"> <img data-src="https://www.techtarget.com/rms/editorial/sAWS_20180510_MigrationHub_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/editorial/sAWS_20180510_MigrationHub_mobile.png 960w,https://www.techtarget.com/rms/editorial/sAWS_20180510_MigrationHub.png 1280w" alt="Screenshot of the AWS Migration Hub dashboard." height="247" width="519"> <figcaption> <i class="icon pictures" data-icon="z"></i>Users can get started with AWS Migration Hub at the dashboard. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p><a href="https://www.techtarget.com/searchitoperations/definition/service-discovery">Resource discovery</a> prior to migration helps to build a business case for migrating and to build a detailed and systematic migration plan. However, discovery is an optional process, so users can start a migration without it. If they choose to bypass the discovery process and migrate immediately, all application and database servers will appear as resources in AWS Migration Hub as the migration takes place.</p> <p>During migration, the servers being migrated appear in the Servers page, which can be accessed from the AWS Migration Hub console navigation pane under Discover. To simplify migration tracking, it's useful to group servers as applications. This can be done from the servers list or from the <a href="https://www.techtarget.com/searchnetworking/feature/Discover-8-network-diagramming-tools-aimed-at-architects">network diagram</a>.</p> <p>Once AWS Application Discovery Service collects data, it stores it in a repository, which AWS Migration Hub accesses. The service then relays that data to the user, who can examine technical specifications along with other performance information.</p> <p>Whichever discovery option a user chooses, they must populate a servers list before grouping servers into an application. After the list populates, the user selects the resources to group into an application. All <a href="https://www.techtarget.com/searchcloudcomputing/tip/An-introduction-to-AWS-IAM-best-practices">AWS Identity and Access Management</a> users within the same account can view each other's applications.</p> </section> <section class="section main-article-chapter" data-menu-title="Track a migration with AWS Migration Hub"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Track a migration with AWS Migration Hub</h2> <p>AWS Migration Hub tracks migration progress status and the details of each server grouped to the application. Specifically, the migration tool communicates the status to AWS Migration Hub at key points during the migration.</p> <p>Users can track migration status by following this procedure:</p> <ul class="default-list"> <li>Navigate to Dashboard in the navigation pane of the Migration Hub console.</li> <li>Select the migration application under Most Recently Updated Applications.</li> <li>The status will show as Not Started, In Progress or Completed.</li> <li>Users can manually change the status from Not Started to In Progress or from In Progress to Completed.</li> </ul> <p>AWS Migration Hub displays the migration status of applications in its console. Each migrating resource shows its status within a diagram and a table. Possible status levels are as follows:</p> <ul class="default-list"> <li>In progress, <a href="https://www.techtarget.com/searchdisasterrecovery/definition/data-replication">replication</a> starting.</li> <li>In progress, replication complete.</li> <li>Completed, <a href="https://www.techtarget.com/searchaws/definition/Amazon-Machine-Image-AMI">Amazon Machine Image</a> created.</li> </ul> <p>Once migration is finished, AWS Migration Hub will also display data on the newly created resources.</p> <p>AWS Migration Hub only tracks resources that are in an AWS repository and are migrated with supported tools. Unsupported tools will not report resource statuses back to AWS Migration Hub.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-1" src="https://www.youtube.com/embed/UEyaTsFTfnk?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="AWS Migration Hub pricing"> <h2 class="section-title"><i class="icon" data-icon="1"></i>AWS Migration Hub pricing</h2> <p>AWS Migration Hub is free to use for these purposes:</p> <ul class="default-list"> <li>Collecting and storing discovery data in the home region.</li> <li>Planning a migration to AWS.</li> <li>Tracking an in-progress migration.</li> </ul> <p>However, organizations can incur costs for using migration tools and for any resources consumed on AWS. Migration Hub's Refactor Spaces is also chargeable on a pay-as-you-use basis at $0.000002 per request. In addition, there are charges for AWS services provisioned by Refactor Spaces, but there are no upfront costs or minimum fees for using Refactor Spaces. AWS also provides a free tier that lets users try Refactor Spaces with 2,160 free environment hours per month for 90 days.</p> <p><em>Organizations should take stock of their applications and modernize them where appropriate as part of a cloud migration. Learn about the <a href="https://www.techtarget.com/searchcloudcomputing/tip/How-to-modernize-apps-as-part-of-the-cloud-migration-process">benefits of modernizing apps and be aware of the potential pitfalls</a>.</em></p> </section> AWS Migration Hub is a service from Amazon Web Services that enables AWS users to simplify migration to the AWS public cloud. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/definition/AWS-Migration-Hub Mon, 19 Aug 2024 12:00:00 GMT <section class="section main-article-chapter" data-menu-title="What is CQRS (command query responsibility segregation)?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is CQRS (command query responsibility segregation)?</h2> <p>CQRS (command query responsibility segregation) is a programming design and architectural pattern that treats retrieving data and changing data differently. CQRS uses command handlers to simplify the <a href="https://www.techtarget.com/searchdatamanagement/definition/query">query</a> process and hide complex, multisystem changes.</p> <p>When the CQRS pattern is combined with the event sourcing pattern, it guarantees an audit <a href="https://www.techtarget.com/whatis/definition/log-log-file">log</a> of changes to the <a href="https://www.techtarget.com/searchdatamanagement/definition/database">database</a> that can help maintain transactional consistency. Moreover, its read model can contain, or be used to generate, a materialized view pattern of write model data.</p> <p>CQRS is a software architecture pattern in which the responsibilities of a system's commands and queries are segregated by vertically slicing the application logic. You can think of the pattern as a way to separate read and update operations for a data store. The goal of CQRS is maximizing application performance, security and scalability.</p> <p>In CQRS, a command conveys the user's intent. It is an instruction the user provides to the system to perform a certain task. A command does not generate a result and is part of the write model. A query is a user's request for information. It returns a result but does not change the state and is part of the read model.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/app_arch-cqrs_pattern-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/app_arch-cqrs_pattern-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/app_arch-cqrs_pattern-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/app_arch-cqrs_pattern-f.png 1280w" alt="diagram showing steps in the command query responsibility segregation pattern" height="241" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Steps in the command query responsibility segregation pattern </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>Greg Young developed the CQRS design pattern in 2010, around the same time as <a href="https://www.theserverside.com/tip/How-the-event-sourcing-design-pattern-works-with-example"><i>event sourcing</i></a> emerged. As he defined the pattern: "[CQRS] uses the same definition of commands and queries that Meyer used and maintains the viewpoint that they should be pure. The fundamental difference is that, in CQRS, objects are split into two objects, one containing the commands, one containing the queries." Young was referring to Bertrand Meyer, who developed CQRS' predecessor, the command query separation (CQS) pattern.</p> </section> <section class="section main-article-chapter" data-menu-title="CQRS vs. CQS"> <h2 class="section-title"><i class="icon" data-icon="1"></i>CQRS vs. CQS</h2> <p>Although related to and inspired by CQS, CQRS is different in that objects are split into two objects: one containing commands and one containing queries. Separate model representations for queries and commands make the approach highly suitable for event sourcing or task-based user interface (<a href="https://www.techtarget.com/searchapparchitecture/definition/user-interface-UI">UI</a>) applications. In contrast, the CQS application architecture pattern is mainly about separating the logic for queries that don't update a system's observable state from the logic of commands that do perform state updates.</p> <p>Bertrand Meyer first referred to CQS in his book, <i>Object-Oriented Software Construction. </i>It is also part of his work on the Eiffel programming language. He proposed separating methods that change state from those that don't. This enables queries to be confidently used in many situations. They can also be introduced anywhere, and their order can be changed as needed.</p> <p><em>Learn more about the <a href="https://www.techtarget.com/searchapparchitecture/tip/A-comparison-of-6-top-programming-languages">most popular programming languages</a> in the world.</em></p> </section> <section class="section main-article-chapter" data-menu-title="When to use the CQRS pattern -- and when not to"> <h2 class="section-title"><i class="icon" data-icon="1"></i>When to use the CQRS pattern -- and when not to</h2> <p>The CQRS pattern is useful in all these scenarios:</p> <ul class="default-list"> <li><b>Collaborative domains.</b> Numerous users can view the same data concurrently.</li> <li><b>Task-based UIs.</b> Users are led through a series of steps to finish a task.</li> <li><b>High-traffic systems.</b> Evenly distributing the workload between read/write operations improves performance and scalability in high-traffic systems.</li> <li><b>Querying from repositories.</b> Query the data users need to view from repositories.</li> <li><b>Complex business logic.</b> Separating read/write processes can help streamline application design if it contains complicated business logic.</li> <li><b>Optimizing read operations.</b> CQRS can create dedicated read models to optimize the read operations, especially if an application has many such operations.</li> <li><b>Different data models for reads/writes.</b> Through CQRS, multiple <a href="https://www.techtarget.com/searchdatamanagement/tip/7-data-modeling-techniques-and-concepts-for-business">data models can be established</a> for each operation by separating the read/write models.</li> <li><b>Support for event sourcing.</b> CQRS can be combined with event sourcing patterns to develop a system that can handle many events and queries.</li> </ul> <p>It's important to consider CQRS' unique requirements and limitations and assess thoroughly whether this pattern is appropriate for its particular use cases. Use it only in restricted situations where it's necessary. It's useless for systems that follow the create, read, update and delete mental model.</p> </section> <section class="section main-article-chapter" data-menu-title="How the CQRS pattern is executed"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How the CQRS pattern is executed</h2> <p>The most common way to deploy CQRS is the command pattern, which is the software system that defines a high-level interface. At runtime, the base class takes the command, creates the appropriate object handler -- perhaps update, delete or create -- and calls a method to execute the command.</p> <p>Before and after execution, the base class can log that the method was called. The log can be replayed from and to any point in time. Once the interface and dispatch code exist, the computer program to take the events and replay them is called a <i>for</i> loop. It reads a file from a certain point, and then, for each line, it calls the command on that line with the data on that line. The complexity hidden inside the command handler is limited or <a href="https://www.techtarget.com/searchnetworking/definition/encapsulation">encapsulated</a> by the handler.</p> <p>The handlers call each step of the process to create, update or delete a logical item in the system. They can dispatch the first request and monitor to see when that action completes, handling errors and rollbacks if needed. The handler sets up a long-running, <a href="https://www.techtarget.com/searchapparchitecture/definition/two-phase-commit-2PC">two-phase commit</a> (<a href="https://www.techtarget.com/searchapparchitecture/tip/Using-the-saga-design-pattern-for-microservices-transactions">saga pattern</a>) post in which it writes the results to the event log. This log contains a record of every system change. If this log is created in a consistent way that a program can read and replay, it enables event sourcing.</p> </section> <section class="section main-article-chapter" data-menu-title="CQRS example: Customer order process"> <h2 class="section-title"><i class="icon" data-icon="1"></i>CQRS example: Customer order process</h2> <p>When a customer looks at an order, a straightforward process is executed: a read from a database. The read could be from a <a href="https://www.techtarget.com/searchdatamanagement/definition/NoSQL-Not-Only-SQL">NoSQL</a> key-value store, like <a href="https://www.computerweekly.com/news/252505466/Whole-system-view-of-railways-from-Resonate-on-Redis-offers-progress">Redis</a>, that stores all customer information. A <a href="https://www.techtarget.com/searchapparchitecture/definition/microservices">microservice</a> picks up the information, and a webpage displays it. The Redis team, working with the front-end developers, can handle the read side easily.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/8bxpZmNKhU8?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> <p>The operation's write side is much more complex and contains several different steps and dependencies. For example, if an order is canceled before it is shipped, the following occurs:</p> <ul class="default-list"> <li>The return must be canceled in the cache and returned in the main source system of record.</li> <li>If there is a <a href="https://www.techtarget.com/searchdatamanagement/definition/data-warehouse">data warehouse</a>, the record must be deleted from it and other auxiliary systems.</li> <li>The physical warehouse and shipping must be notified to not ship the item.</li> <li>Credit card charges must be reversed, inventory counts in the warehouse must be changed and the supplier's refill order must go down by one.</li> </ul> <p>Since a single change (order cancellation) must be copied to so many places, the update, delete and create sides of the operation must interact with either an enterprise service bus (<a href="https://www.techtarget.com/searchapparchitecture/definition/Enterprise-Service-Bus-ESB">ESB</a>)or a command handler, such as the saga pattern. CQRS helps create the logic to handle these complex transactions.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineImages/data_management-ods_vs_warehouse.jpg"> <img data-src="https://www.techtarget.com/rms/onlineImages/data_management-ods_vs_warehouse_mobile.jpg" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineImages/data_management-ods_vs_warehouse_mobile.jpg 960w,https://www.techtarget.com/rms/onlineImages/data_management-ods_vs_warehouse.jpg 1280w" alt="A chart showing key differences between an operational data store and a data warehouse" height="241" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>An operational data store has important differences from a data warehouse, including requirements for how CQRS works with them. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Challenges of CQRS"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Challenges of CQRS</h2> <p>Some challenges associated with CQRS are the following:</p> <ul class="default-list"> <li><b>Complexity.</b> CQRS creates complexity on top of the codebase. Instead of a simple relational database, like a <a href="https://www.techtarget.com/searchdatamanagement/definition/SQL">Structured Query Language</a> database, the application now has command handlers, dispatching and logging, all of which increase the potential for error.</li> <li><b>Messaging.</b> CQRS doesn't require messaging, but it's typical to use messaging to process commands and broadcast update events, with the application having to handle message failures or duplicate messages.</li> <li><b>Conversion.</b> It can be challenging to convert to CQRS currently online running noncomplex systems like those that don't have a complex transition log or systems that use overnight batch mode to do updates.</li> <li><b>Eventual consistency.</b> Separating the read/write databases can cause the read data to become stale. Updating the former to reflect changes in the latter can cause data consistency issues. It can be difficult to detect when a user issued a request based on outdated data from the reading database.</li> <li><b>Delayed data synchronization.</b> Changes made in the write mode can take some time to reflect in the read mode, which can <a href="https://www.computerweekly.com/feature/Can-real-time-data-visualisation-deliver-trust-and-opportunity">cause issues</a> if a user queries the read mode before change synchronization.</li> <li><b>Cost.</b> CQRS patterns can increase hardware costs. <a href="https://www.techtarget.com/searchcloudcomputing/opinion/Utilize-AI-for-cloud-cost-optimization">Cloud utilization costs</a> can also increase since additional database technologies are required for deployment.</li> </ul> <div class="youtube-iframe-container"> <iframe id="ytplayer-1" src="https://www.youtube.com/embed/cdzUXv8SpjY?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Combining CQRS with other patterns: CQRS and event sourcing"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Combining CQRS with other patterns: CQRS and event sourcing</h2> <p>The CQRS pattern is frequently used with the event sourcing pattern. The latter is an approach to a series or sequence of events. Events are user-generated actions, such as clicking a mouse button or pressing a keyboard key. CQRS-based systems use separate read/write data models, where each model is tailored to relevant tasks and often located in physically separate data stores.</p> <p>In CQRS, the system tracks past changes in data states and alerts users on how to handle them. Every step of the data transformation process is recorded, not just the most recent state. Since the system itself can be recreated as the collection of all the changes over time, it creates a new view of data in the system: the transaction log. This log is not just a text file, but an actual set of commands that can be replayed to recreate the change. In some cases, the event log can be "played backward" to undo large changes.</p> <p>Setting up event sourcing requires capturing the changes, along with building the software to replay those changes. Deployed carefully, command handlers can write to a log, creating the transaction log for event sourcing. Running each line in the log through a command handler executes event sourcing.</p> <p>The CQRS and event sourcing patterns have considerable overlap. The command in CQRS is one way to create event sourcing. CQRS works well with event sourcing since it divides the system by business operations and events are the result of those operations.</p> </section> <section class="section main-article-chapter" data-menu-title="What are some related patterns?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What are some related patterns?</h2> <p>CQRS is compatible with <a href="https://www.techtarget.com/whatis/definition/domain-driven-design">domain-driven design</a> -- an explicit domain model -- event sourcing and the command pattern. An ESB might deliver the messages to perform the create, update and delete. The general pattern is <a href="https://www.techtarget.com/searchapparchitecture/definition/service-oriented-architecture-SOA">service-oriented architecture</a>.</p> <p><em>While CQRS can be useful in constructing an event-driven architecture, bad practices can turn it into more of a nuisance. Explore <a href="https://www.techtarget.com/searchapparchitecture/tip/Common-CQRS-pattern-problems-and-how-teams-can-avoid-them">common CQRS pattern problems</a>, and see how they can be mitigated.</em></p> </section> CQRS (command query responsibility segregation) is a programming design and architectural pattern that treats retrieving data and changing data differently. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchapparchitecture/definition/CQRS-command-query-responsibility-segregation Fri, 31 May 2024 09:00:00 GMT <section class="section main-article-chapter" data-menu-title="What are AWS Reserved Instances?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What are AWS Reserved Instances?</h2> <p>AWS Reserved Instances (RIs) refer to compute capacity that can be reserved on Amazon Elastic Compute Cloud (<a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Compute-Cloud-Amazon-EC2">Amazon EC2</a>) and Amazon Relational Database Service (<a href="https://www.techtarget.com/searchaws/definition/Amazon-Relational-Database-Service-RDS">Amazon RDS</a>) to take advantage of discounted pricing. Rather than a physical instance, an RI is a billing discount that can be applied to all types of physical instances.</p> <p>To create RIs, companies choose one of the various instance types available in an Amazon Web Services (AWS) service and an associated operating system (<a href="https://www.techtarget.com/whatis/definition/operating-system-OS">OS</a>), term, tenancy specification, Region and <a href="https://www.techtarget.com/searchaws/definition/availability-zones">Availability Zone</a> (AZ). The payment option that AWS will consider during billing is also selected. In some cases, RIs can be exchanged within the instance family or moved between different Availability Zones within the Region.</p> </section> <section class="section main-article-chapter" data-menu-title="What is a Reserved Instance in AWS?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is a Reserved Instance in AWS?</h2> <p>An instance is a server resource provided by AWS. It contains all the compute resources such as <a href="https://www.techtarget.com/searchitoperations/definition/container-containerization-or-container-based-virtualization">containers</a>, databases, <a href="https://www.techtarget.com/searchapparchitecture/definition/microservices">microservices</a> and <a href="https://www.techtarget.com/searchitoperations/definition/virtual-machine-VM">virtual machines</a> that organizations need to run their workloads in the AWS cloud. AWS maintains all the necessary hardware in its data centers and provides virtual access to the compute resources in the form of an instance. Different types of workloads can be run on AWS instances, including compute-intensive, memory-intensive, storage-intensive and <a href="https://www.techtarget.com/searchdatacenter/definition/high-performance-computing-HPC">high-performance computing</a> workloads.</p> <p>A Reserved Instance is not a physical instance. Rather, it refers to a billing discount offered when organizations reserve capacity in an AWS service. The discount is applied to the use of on-demand instances in the AWS customer's account.</p> </section> <section class="section main-article-chapter" data-menu-title="Benefits of AWS Reserved Instances"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Benefits of AWS Reserved Instances</h2> <p>RIs provide users with the confidence to launch instances when they need them. At the same time, they can help users save on the cost of equivalent <a href="https://www.techtarget.com/searchdatacenter/definition/capacity-on-demand">on-demand capacity</a>. For example, RIs on Amazon EC2 can provide cost savings of up to 72% over on-demand instances. Reserving capacity also enables organizations to manage their cloud budgets more predictably, minimize risks and comply with policies that require longer-term commitments.</p> <p>The costs and usage associated with RIs can be easily viewed and managed using three AWS cost management tools:</p> <ul class="default-list"> <li>Default RI reports in AWS Cost Explorer.</li> <li>Reservation-related data in the Cost and Usage Report.</li> <li>RI purchase recommendations provided by AWS Cost Explorer.</li> </ul> </section> <section class="section main-article-chapter" data-menu-title="Payment options for Reserved Instances in AWS"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Payment options for Reserved Instances in AWS</h2> <p>RIs are available on two AWS services: Amazon EC2 and Amazon RDS. For both services, AWS offers three options to pay for the RIs:</p> <ul class="default-list"> <li>All upfront (AURI).</li> <li>Partial upfront (PURI).</li> <li>No upfront payments (NURI).</li> </ul> <p>All three options have their pros and cons.</p> <p>The AURI option offers the largest discount, but since it involves a larger upfront investment, it might not be suitable for every organization's tech budget. With PURI, companies spend less upfront, but also get smaller discounts. Finally, NURI is best for companies that prefer not to spend anything upfront. This way, they can free up capital for other purposes, even if they save the least amount of money on RIs over time.</p> <p>With the PURI option, the remaining balance becomes due in monthly increments over the term, while with NURI, users make all payments in monthly increments over the term.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/-xanQ3aUWms?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Reserved Instances in Amazon EC2"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Reserved Instances in Amazon EC2</h2> <p>Amazon EC2 RIs are best for companies looking to save significant amounts of money -- up to 72% -- compared with the costs incurred with on-demand instances. These RIs are available at a discounted hourly rate, either when applied to all the <a href="https://www.techtarget.com/searchcloudcomputing/tip/Understand-AWS-Regions-vs-Availability-Zones">AZs in an AWS Region</a> or when used in a specific AZ. The billing is automatically adjusted to apply the discounted rate when the attributes of EC2 instance usage match the attributes of an active RI, such as the instance type, platform or tenancy.</p> <p>Also, the RI's capacity reservation is automatically used by any running instances matching its attributes. The RIs can be shared between multiple accounts within a consolidated billing family.</p> <p>Three types of RIs are available with Amazon EC2:</p> <ul class="default-list"> <li><b>Standard.</b> These RIs provide the highest discount of up to 40% with a one-year commitment and up to 60% with a three-year commitment. They are most suitable for applications that require steady-state instance usage.</li> <li><b>Convertible.</b> Convertible RIs provide slightly lower discounts -- up to 31% with a one-year commitment and up to 54% with a three-year commitment -- than standard RIs. However, enterprise users that reserve EC2 capacity with convertible RIs can change instance families, OS types and tenancies. Convertible RIs are also best for steady-state usage.</li> <li><b>Scheduled.</b> These RIs can be launched within specific reserved time windows, even if the windows are only a fraction of a day, week or month.</li> </ul> <p>Both standard and convertible EC2 RIs require a one-year or three-year commitment from the customer. One key difference between them is that with the convertible RI, changes are allowed to AZ, instance size, networking type and instance families, OS, tenancy, and payment options. With the standard RI, only changes to AZ, instance size and networking type are allowed. Also, convertible RIs can be exchanged for different convertible RIs of equal or greater value. This cannot be done with standard or scheduled RIs.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/amazon_ec2_pricing_options-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/amazon_ec2_pricing_options-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/amazon_ec2_pricing_options-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/amazon_ec2_pricing_options-f.png 1280w" alt="Amazon EC2 pricing options table." height="314" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Reserved Instances for Amazon EC2 help companies save significant amounts of money compared with using on-demand instances. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Amazon EC2 RI Marketplace"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Amazon EC2 RI Marketplace</h2> <p>When companies purchase Amazon EC2 RIs from AWS, either through the <a href="https://www.techtarget.com/searchaws/definition/AWS-Management-Console">AWS Management Console</a> or using application programming interface tools, they must commit to either a one-year or three-year term. Also, the purchases are nonrefundable.</p> <p>Organizations also have the option of buying RIs with shorter terms from Reserved Instance Marketplace sellers. These third-party RIs are usually of the standard type. When a customer makes such a purchase, AWS shares the buyer's city, state and ZIP code with the seller for tax purposes. If the buyer no longer needs the RI, they can either modify or exchange it, or list it -- for standard RIs only -- on the Reserved Instance Marketplace.</p> </section> <section class="section main-article-chapter" data-menu-title="Reserved Instances in Amazon RDS"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Reserved Instances in Amazon RDS</h2> <p>RIs are also available in Amazon RDS, again for a one-year or three-year period. As with Amazon EC2, RDS RIs enable significant cost savings -- up to 69% -- compared with on-demand RDS instance pricing when used in a steady state. The RI discounts are tied to instance type and AWS Region.</p> <p>To use an Amazon RDS Reserved Instance, a new <a href="https://www.techtarget.com/searchdatamanagement/definition/database">database</a> instance must first be created, similar to how it's done for using an on-demand instance. The new DB instance must match the Reserved Instance in all these aspects: AWS Region; DB engine; DB instance type and size; edition; and license type.</p> <p>In Amazon RDS, all RI types are available for these database engines:</p> <ul class="default-list"> <li><a href="https://www.techtarget.com/searchaws/definition/Amazon-Aurora">Amazon Aurora</a>.</li> <li>MariaDB.</li> <li><a href="https://www.techtarget.com/searchoracle/definition/MySQL">MySQL</a>.</li> <li><a href="https://www.techtarget.com/searchoracle/definition/Oracle">Oracle</a>.</li> <li>PostgreSQL.</li> <li><a href="https://www.techtarget.com/searchdatamanagement/definition/SQL-Server">SQL Server</a>.</li> </ul> <p>In Aurora, MariaDB, MySQL and PostgreSQL, users have instance size flexibility. The bring-your-own-license edition of Oracle also offers similar flexibility.</p> <p>All three payment options -- AURI, PURI and NURI -- are available for Amazon RDS RIs. However, the NURI option is only available for a one-year term. The AURI and PURI options are available for both one-year and three-year terms.</p> <p><em>AWS users should take steps to avoid unnecessary charges and keep cloud costs in check. Read here to <a href="https://www.techtarget.com/searchcloudcomputing/tip/Get-a-handle-on-AWS-costs-with-these-tips-and-tools">get a handle on AWS costs with these tips and tools</a>.</em></p> </section> AWS Reserved Instances (RIs) refer to compute capacity that can be reserved on Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Relational Database Service (Amazon RDS) to take advantage of discounted pricing. https://cdn.ttgtmedia.com/visuals/digdeeper/1.jpg https://www.techtarget.com/searchaws/definition/AWS-Reserved-Instances-Amazon-Reserved-Instances Tue, 14 May 2024 11:00:00 GMT <section class="section main-article-chapter" data-menu-title="What is a fact table?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is a fact table?</h2> <p>In <a href="https://www.techtarget.com/searchdatamanagement/definition/data-warehouse">data warehousing</a>, a fact table is a database table in a dimensional model. The fact table stores quantitative information for analysis. The table lies at the center of the dimensional model, surrounded by multiple <a href="https://www.techtarget.com/searchdatamanagement/definition/dimension-table">dimension tables</a>. Each dimension table contains a set of related <a href="https://www.techtarget.com/whatis/definition/attribute">attributes</a> that describe the facts in the fact table. Together with the <a href="https://www.techtarget.com/searchdatamanagement/definition/dimension">dimensions</a>, the fact table provides a consolidated view of the quantitative and attribute data, which is optimized for querying, reporting and <a href="https://www.techtarget.com/searchbusinessanalytics/definition/big-data-analytics">analytics</a>.</p> <p>A fact table usually contains two types of columns: measures and <a href="https://www.techtarget.com/searchoracle/definition/foreign-key">foreign keys</a>. The measures contain the quantitative data, usually in the form of numeric values. The measures represent key <a href="https://www.techtarget.com/searchcustomerexperience/definition/business-metric">business metrics</a> -- or facts -- about a <a href="https://www.techtarget.com/searchcio/definition/business-process">process</a>, event or condition. The fact table's <a href="https://www.techtarget.com/searchdatamanagement/definition/primary-key">primary key</a> is usually a composite of the foreign key columns, although it can also be a surrogate key.</p> <p>The foreign key columns join to multiple dimension tables and their attributes, making it possible to "slice and dice" the warehouse data by various attribute combinations to answer specific business questions. For example, key <a href="https://www.techtarget.com/searchcio/definition/stakeholder">stakeholders</a> in an organization might want to answer the question: "How many female customers in Utah and Nevada between ages 42 through 62 purchased hiking boots in May and June 2023?"</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/data_management-database_keys.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/data_management-database_keys_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/data_management-database_keys_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/data_management-database_keys.png 1280w" alt="Common database keys explained." height="224" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>There are many common database keys including primary key, candidate key, super key and foreign key. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Data warehouse layout"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Data warehouse layout</h2> <p>A data warehouse is typically laid out in a <a href="https://www.techtarget.com/searchdatamanagement/definition/star-schema">star schema</a> or <a href="https://www.techtarget.com/searchdatamanagement/definition/snowflaking">snowflake schema</a>, with the fact table at the center. A data warehouse can contain multiple fact tables, but each of those tables still lies at the center of its respective dimensions.</p> <p>The descriptive attributes in the dimension tables make it possible to filter, categorize and summarize the facts to extract the answers to critical business questions. The following figure shows a simple star schema with four dimension tables and one fact table.</p> <p>The fact table lies at the heart of this <a href="https://www.techtarget.com/searchdatamanagement/definition/data-modeling">data model</a>. It includes five measures (UnitPrice, SalesAmount, UnitsSold, PercentProfit and DailyInventory) as well as multiple foreign keys that <a href="https://www.techtarget.com/searchdatamanagement/answer/What-are-the-differences-between-fact-tables-and-dimension-tables-in-star-schemas">reference the dimension tables</a>. The foreign keys make it possible to access the dimensions' attributes, which contain information about the customers, products, territories and dates involved in each sale. The <a href="https://www.techtarget.com/searchdatamanagement/feature/Data-lake-vs-data-warehouse-Key-differences-explained">warehouse data</a> can be filtered by any combination of attributes available in the dimension tables.</p> <p>Four of the foreign keys reference the same date dimension, making it possible to slice the data by any of those dates. Some dimension tables, such as dimTerritory and dimProduct, contain hierarchical data, which can be broken down into more granular segments. For example, stakeholders might request a report that shows the total annual sales by product type and sales territory. Because of the hierarchical nature of the data, they'll also be able to drill down to view sales totals for individual products and countries.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/a_basic_star_schema_example-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/a_basic_star_schema_example-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/a_basic_star_schema_example-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/a_basic_star_schema_example-f.png 1280w" alt="Example diagram of a star schema with a fact table." height="353" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Star schema with four dimension tables (blue) and one fact table (green). </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="Types of fact tables"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Types of fact tables</h2> <p>Fact tables serve different purposes and operate at different grains. The grain determines what a record in the <a href="https://www.techtarget.com/whatis/definition/table">table</a> should represent at its most atomic level. The table might include records at other grains, such as inventory totals, but the table's core purpose is to host records based on the defined grain. In the previous example, the fact table's grain can be specified as the "sale of a product to a customer at a specific time and place." A fact table's design should be based on the lowest possible and practical grain.</p> <p>Fact tables are often categorized as one of the following types:</p> <ul class="default-list"> <li><b>Transactional.</b> This is the most basic type of fact table and the most used. The table's grain is typically treated as one row per transaction or event. In the example, the factSales table is a <a href="https://www.techtarget.com/whatis/definition/transactional-data">transactional</a> fact table. Each row records one sales transaction.</li> <li><b>Periodic snapshot.</b> The fact table stores a snapshot of the dimensional data at a defined period of time, such as capturing weekly or monthly sales totals.</li> <li><b>Accumulating snapshot.</b> The fact table stores snapshots of a process or activity that has a specific beginning and end. For example, this type of fact table might be used to track service calls or the order processing cycle.</li> </ul> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/data_architect_roles_and_responsibilities-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/data_architect_roles_and_responsibilities-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/data_architect_roles_and_responsibilities-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/data_architect_roles_and_responsibilities-f.png 1280w" alt="List of data architect roles and responsibilities." height="288" width="559"> <figcaption> <i class="icon pictures" data-icon="z"></i>A data architect's roles and responsibilities are great and include research, evaluation, integration, deployment, monitoring and collaboration. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>Fact tables can also include <a target="_blank" href="https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/additive-semi-additive-non-additive-fact/" rel="noopener">different types</a> of measures, which provide the quantitative data needed to find answers to specific business questions. A fact table supports three types of measures:</p> <ul class="default-list"> <li><b>Additive.</b> The measure can be added across any dimension. In the previous example, the fact table's UnitsSold column can be summed up based on customer, product, date, territory or any combination of these. As a result, the column is considered an additive measure. Additive measures are the most common and easiest to work with.</li> <li><b>Non-additive.</b> The measure cannot be added across any dimensions. In the example, the PercentProfit column stores the percentage of profit on each sale. If those percentages are added up across a dimension, you might end up with totals such as 26,000%. Because the percentages should not be added together, the column is considered a non-additive measure.</li> <li><b>Semi-additive.</b> The measure can be added across some dimensions but not others. In the example, the DailyInventory column maintains a record of the number of products in stock on each day. Although it's possible to aggregate these totals for products and territories, it does not make sense to add them together for the dimDate dimension. For instance, if there were 25 widgets in stock on July 27, and 25 widgets on July 28, it does not mean that there are a total of 50 widgets in stock.</li> </ul> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/TcgiZFZ-r1U?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> <p>It is also possible to create fact tables that contain no measures. These tables, referred to as factless fact tables, are basically collections of foreign keys that join to dimension tables. Even without measures, this type of fact table can still <a href="https://www.techtarget.com/searchbusinessanalytics/tip/6-essential-big-data-best-practices-for-businesses">provide useful information</a>. For example, if the fact table in the example included no measures, the customers, products, territories and dates could still be linked together to provide a complete picture of all sales events, which could be filtered along any of the dimensions.</p> <p>When designing fact tables and their related dimension tables, <a href="https://www.techtarget.com/searchdatamanagement/definition/data-architect">data architects</a> should take into account both current and future needs, ensuring that <a href="https://www.techtarget.com/searchdatamanagement/tip/9-steps-to-a-dynamic-data-architecture-plan">the design is flexible</a> enough to accommodate changing business requirements without having to rebuild the warehouse.</p> <p><em>Evaluate <a href="https://www.techtarget.com/searchdatamanagement/feature/Evaluating-your-need-for-a-data-warehouse-platform">data warehouse deployment options and use cases</a>. Explore the <a href="https://www.techtarget.com/searchdatamanagement/answer/What-are-the-differences-between-fact-tables-and-dimension-tables-in-star-schemas">differences between dimension tables vs. fact tables</a>. Check out the <a href="https://www.techtarget.com/searchdatamanagement/tip/On-premises-vs-cloud-data-warehouses-Pros-and-cons">pros and cons of on-premises vs. cloud data warehouses</a>. Read about <a href="https://www.techtarget.com/searchdatamanagement/feature/5-data-integration-challenges-and-how-to-overcome-them">data integration challenges and how to overcome them</a> and see how to <a href="https://www.techtarget.com/searchdatamanagement/tip/Developing-an-enterprise-data-strategy-10-steps-to-take">develop an enterprise data strategy</a>.</em></p> </section> In data warehousing, a fact table is a database table in a dimensional model. The fact table stores quantitative information for analysis. https://cdn.ttgtmedia.com/visuals/digdeeper/5.jpg https://www.techtarget.com/searchdatamanagement/definition/fact-table Wed, 27 Dec 2023 11:30:00 GMT <p>While Amazon EC2 provides the ability to launch a variety of virtual servers, there is also the need to deploy data storage for each virtual server.</p> <p>Amazon Elastic Block Storage (<a href="https://www.techtarget.com/searchaws/definition/Amazon-EBS-Amazon-Elastic-Block-Store">EBS</a>) volumes are virtual storage devices in the cloud that get attached to <a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Compute-Cloud-Amazon-EC2">EC2</a> instances and provide internal storage for essential areas in any application, such as OS files, source code, libraries and configuration files. Given the critical nature of this data, enterprises need a reliable way to create and manage backups of EBS volumes.</p> <p>Learn how EBS snapshots can help with disaster recovery, as well as how to create and manage them through the AWS Command Line Interface.</p> <section class="section main-article-chapter" data-menu-title="EBS snapshot features and pricing"> <h2 class="section-title"><i class="icon" data-icon="1"></i>EBS snapshot features and pricing</h2> <p>EBS snapshots are a point-in-time backup of an EBS volume. With EBS snapshots, users can control how data in an EBS volume is backed up, stored, managed and recovered. Developers can trigger the creation of an EBS snapshot using the AWS CLI or SDK, or through the EC2 console. Some core features of EBS snapshots include the following:</p> <ul class="default-list"> <li><b>Direct read access.</b> Users can read snapshots' data and pinpoint differences between two EBS snapshots without needing to create EBS volumes and EC2 instances via the EBS direct API. This feature enables users to better track incremental changes on EBS volumes, which provides increased granularity and faster backup times.</li> <li><b>Create snapshots from any block storage.</b> In addition, with the direct API, users can create snapshots directly from any block storage data, no matter where it is stored, and then recover into EBS volumes.</li> <li><b>Copy snapshots across AWS Regions.</b> With EBS, users can automatically copy snapshots <a href="https://www.techtarget.com/searchcloudcomputing/tip/Understand-AWS-Regions-vs-Availability-Zones">across other Regions</a>. This helps migration and disaster recovery efforts.</li> <li><b>Share snapshots.</b> Users have the choice to share data within their organization as well as publicly to all AWS users.</li> </ul> <p>EBS snapshots are a better fit for the creation of regular backups of EBS volumes. For a solid disaster recovery strategy, users should create snapshots routinely and copy them to other AWS Regions and AWS accounts. EBS snapshots can also be transitioned to cold storage, which is an option to consider to significantly reduce storage costs.</p> <h3>EBS snapshot pricing</h3> <p>EBS snapshots cost $0.05 per GB, per month, for the standard tier. The cost to create and keep an <a href="https://www.techtarget.com/searchaws/definition/Amazon-Machine-Image-AMI">Amazon Machine Image</a> depends solely on the EBS snapshots that are essential to that AMI. A 100 GB EBS snapshot or AMI would cost $5 per month. Subsequent snapshots of a particular EBS volume are charged based only on the blocks that have changed since the previous snapshot.</p> <p>To reduce costs, users can transition snapshots to the Archive tier, which costs $0.0125 per GB, per month. In the Archive tier, the same 100 GB snapshot would then cost $1.25 per month. Data retrieval from the Archive tier is billed at $0.03 per GB. Be aware that there's a minimum 90-day retention period.</p> </section> <section class="section main-article-chapter" data-menu-title="EBS snapshots vs. Amazon Machine Images"> <h2 class="section-title"><i class="icon" data-icon="1"></i>EBS snapshots vs. Amazon Machine Images</h2> <p>When users create an EBS snapshot, they can use it to configure an EC2 AMI. An AMI supplies a storage image, which a new EC2 instance requires at launch.</p> <p>An active EC2 instance can also create an AMI. The EC2 service first creates one or more EBS snapshots, which are then automatically used for the new AMI. The number of snapshots produced depends on how many EBS volumes are attached to the source EC2 instance. The EC2 service creates one EBS snapshot per attached EBS volume.</p> <p>EC2 instances can be launched only from an existing AMI and not directly from an EBS snapshot. If an EBS snapshot is used as the source data to launch new EC2 instances, it must first be used to create an AMI. Then, the AMI will be the input when launching new EC2 instances.</p> <p>Since AMIs are a way to manage the starting point of new EC2 instances, there is no need to create AMIs as frequently as EBS snapshots. Also, AWS recommends maintaining cross-region and cross-account copies of AMIs for disaster recovery purposes, as well as enabling encryption when creating EBS volumes and snapshots.</p> </section> <section class="section main-article-chapter" data-menu-title="Create and manage EBS snapshots"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Create and manage EBS snapshots</h2> <p>It's convenient to launch and manage EBS snapshots through the AWS CLI. The <span style="font-family: 'courier new', courier, monospace;">create-snapshot</span> command provides a way to quickly start snapshot creation. A basic example of this CLI command looks like the following example.</p> <pre><span style="font-family: 'courier new', courier, monospace;">aws ec2 create-snapshot --volume-id &lt;ebs-volume-id&gt; –description &lt;ebs-snapshot-description&gt;</span></pre> <p>This command returns, among other data, the <span style="font-family: 'courier new', courier, monospace;">SnapshotId</span> value of the recently created snapshot, as well as its creation state, the value of which will be <span style="font-family: 'courier new', courier, monospace;">Pending</span> right after calling this command.</p> <p>The <span style="font-family: 'courier new', courier, monospace;">describe-snapshots</span> command in the following example offers a way to list and filter snapshots in your account.</p> <pre><span style="font-family: 'courier new', courier, monospace;">aws ec2 describe-snapshots --snapshot-ids</span> <span style="font-family: 'courier new', courier, monospace;">&lt;space-separated-list-of-snapshot-ids&gt;</span></pre> <p>This command also offers a useful <span style="font-family: 'courier new', courier, monospace;">filters</span> parameter to narrow the list of returned EBS snapshots.</p> <p>As with any AWS CLI command, it's important to define <span style="font-family: 'courier new', courier, monospace;">region</span> and <span style="font-family: 'courier new', courier, monospace;">profile</span> parameters, or set them through the <span style="font-family: 'courier new', courier, monospace;">AWS_DEFAULT_REGION</span> and <span style="font-family: 'courier new', courier, monospace;">AWS_PROFILE</span> <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html" target="_blank" rel="noopener">environment variables</a>, respectively. The examples provided show basic scenarios. Keep in mind that the CLI offers additional parameters that deliver more options.</p> </section> EBS snapshots are an essential part of any data backup and recovery strategy in EC2-based deployments. Become familiar with how to create and manage them with the AWS CLI. https://cdn.ttgtmedia.com/rms/onlineimages/container_g1294273513.jpg https://www.techtarget.com/searchcloudcomputing/tutorial/How-to-create-and-manage-Amazon-EBS-snapshots-via-AWS-CLI Wed, 12 Jul 2023 12:12:00 GMT <p>Delays in application delivery not only hurt the bottom line, they provide a negative customer experience.</p> <p>Placement within the AWS global infrastructure affects how well an application performs. Still, it is an oversimplification to say that the distance between the client and the server is the cause of network <a href="https://www.techtarget.com/searchnetworking/video/A-brief-explanation-of-latency">latency</a>.</p> <p>Data travels through fiber optic cables at the speed of light. Unfortunately, the <a href="https://www.techtarget.com/searchnetworking/definition/TCP-IP">TCP/IP protocol</a> does not employ direct, uninterrupted connections. Instead, requests hop from router to router, with data packets read and rerouted at each stop.</p> <p>For example, an HTTP request from Toronto to the TechTarget server in Boston required seven hops to complete. These hops create latency.</p> <figure class="main-article-image half-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/seven_hops_powershell-h.jpg"> <img data-src="https://www.techtarget.com/rms/onlineimages/seven_hops_powershell-h_half_column_mobile.jpg" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/seven_hops_powershell-h_half_column_mobile.jpg 960w,https://www.techtarget.com/rms/onlineimages/seven_hops_powershell-h.jpg 1280w" alt="An example of network latency and tracing a route" height="121" width="280"> <figcaption> <i class="icon pictures" data-icon="z"></i>An example of route tracing a request from Toronto to TechTarget's East coast servers. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>Amazon offers IT teams the following ways to design high-performance, low-latency architectures:</p> <ul class="default-list"> <li><b>AWS Regions.</b> Provide a full suite of compute, data storage and serverless capabilities.</li> <li><b>AWS Local Zones.</b> Provide compute power near high-density populations.</li> <li><b>AWS edge locations.</b> Quicken the delivery of cacheable resources to customers around the world.</li> <li><b>AWS Outposts.</b> Allows companies to install AWS hardware inside their own data center.</li> </ul> <p>With an understanding of the global distribution of your customer base, your application's latency requirements and your project's budget, these options can be optimally combined to build an architecture that best suits your cloud computing needs.</p> <p>But how do you know if AWS Local Zones is the right choice for your low-latency app? Learn more about Local Zones and its uses, as well as how it compares with other options.</p> <section class="section main-article-chapter" data-menu-title="AWS Local Zone benefits and use cases"> <h2 class="section-title"><i class="icon" data-icon="1"></i>AWS Local Zone benefits and use cases</h2> <p>Amazon builds AWS Local Zones near densely populated city centers to reduce the round-trip distance data must travel to fulfill nearby users' web-based requests. For example, a Dallas user in the AWS Local Zone in Dallas cuts more than 2,000 miles from the roundtrip distance packets must travel versus the parent region's data center in Northern Virginia. Since these zones are placed in close proximity to the clients who use them, low-latency applications can meet rigorous service-level agreements requirements.</p> <p>The response time of a web-based request is a combination of the following two factors:</p> <ul class="default-list"> <li><b>Origin time.</b> How long it takes the server to process the request.</li> <li><b>Round-trip time.</b> How long a request takes to travel back and forth across the network.</li> </ul> <p>A well-designed AWS architecture, built upon technologies like Amazon <a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Compute-Cloud-Amazon-EC2">EC2</a>, Amazon <a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Container-Service-for-Kubernetes-Amazon-EKS">EKS</a> and serverless functions, can bring the origin time down to less than 10 milliseconds. At these service levels, total response time becomes entirely dependent upon how long it takes data to travel across the network.</p> <p>AWS Local Zones reduce the number of network hops required to route a request and provide high-speed, private access to the Amazon network backbone.</p> <p>A Local Zone also reduces the number of hops between the Local Zone, the closest Availability Zone (AZ) and systems that run in other AWS Regions around the globe.</p> <p>With per-usage rates that are only marginally higher than the AZ hosted equivalent, AWS Local Zones are an effective, budget-conscious way to reduce response times for latency-sensitive applications. For applications that require extremely low latency but must also access block storage or microservices hosted in remote AWS regions, the use of an AWS Local Zone makes sense.</p> <h3>Use cases</h3> <p>High-frequency trading is a compelling use-case for AWS Local Zones. For every city in the Western world that has a high-volume stock exchange, Amazon has either <a href="https://aws.amazon.com/about-aws/global-infrastructure/localzones/locations/" target="_blank" rel="noopener">announced or implemented</a> a fully functional nearby AWS Local Zone. The majority of AWS Local Zones provide support for AWS compute-related services such as EC2, EBS, EKS and more.</p> <p>Other examples of applications that benefit from reduced latency and the local compute capacity an AWS Local Zone provides include:</p> <ul class="default-list"> <li>Medical automation.</li> <li>Military and policing.</li> <li>Real-time gambling.</li> <li>Online auctions.</li> <li>Virtual reality gaming.</li> <li>Cloud-based IDEs and editors.</li> </ul> <h3>Drawbacks</h3> <p>Persistence services such as block storage, relational systems and NoSQL databases are generally not available in Local Zones. To access any AWS Service other than the limited set offered through the Local Zone, enterprises will need a more remote AZ.</p> <p>Most applications require access to persistent storage services such as Amazon S3 and Amazon Relational Databases (RDS). Most applications require access to persistent storage services, such as S3 or RDS, so hosting any application in a Local Zone that requires constant access to data stored in a remote AZ would defeat the purpose. After all, why not host everything in the more remote AZ if every request fetches S3 or RDS data stored there?</p> <p>This is where network latency and the way the AWS backbone connects Local Zones to their parent region come into play.</p> </section> <section class="section main-article-chapter" data-menu-title="Compare Local Zones with other AWS options"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Compare Local Zones with other AWS options</h2> <p>AWS has several options to place resources close to densely populated areas, which are comparable to Local Zones.</p> <h3>CDNs</h3> <p>Content delivery networks (<a href="https://www.techtarget.com/searchnetworking/definition/CDN-content-delivery-network">CDNs</a>) such as Cloudflare, Akamai and Amazon <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/HowCloudFrontWorks.html" target="_blank" rel="noopener">CloudFront</a> also place resources closer to those who access them. A CDN caches static content at more than 150 edge locations. AWS CloudFront routes HTTP traffic to the edge location closest to the client making the request. CloudFront greatly reduces download times for shared files that rarely change, such as images, videos, CSS files and HTML pages.</p> <p>While CloudFront and Local Zones solve the latency problem through servers, the specific use case each service addresses is quite different.</p> <h3>Edge locations</h3> <p>There's a key difference between CloudFront edge locations and AWS Local Zones. CloudFront will serve shared, static, cached content from any one of more than 150 edge locations around the world, while a Local Zone is a single, redundantly designed data center located close to large, urban populations with the ability to provide AWS compute services.</p> <p>Unlike an AWS Local Zone, edge locations do not allow users to host applications, <a href="https://www.techtarget.com/searchitoperations/tip/Strategies-for-Kubernetes-multi-cluster-management">manage Kubernetes clusters</a> or run serverless Lambda functions. In contrast, AWS Local Zones support services like ECS and EC2.</p> <p>Webpage metrics like time-to-first load or time-to-first byte are improved using edge locations. CloudFront cannot improve response times for applications that require compute power.</p> <p>If the need is to improve response times for low latency, data processing applications that require application logic, only AWS Local Zones meet the requirements.</p> <h3>Availability Zones</h3> <p>An AWS Local Zone provides more services than an edge location, but far fewer than an AWS Region's AZ. Local Zones are only guaranteed to host compute offerings such as EC2, ECS and EKS services. When accessed through a Local Zone, compute services come with a slightly higher cost than equivalent ones hosted in the Region's AZ.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/aws_availability_zones_vs_regions-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/aws_availability_zones_vs_regions-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/aws_availability_zones_vs_regions-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/aws_availability_zones_vs_regions-f.png 1280w" alt="Multiple availability zones make up an AWS region." height="370" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>An AWS Region is made up of multiple Availability Zones. Users can configure a Region to extend to an AWS Local Zone. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>For example, a Windows-based, t3.xlarge system with 16 gibibytes (GiB) of memory would cost $0.24 per hour if hosted in an Oregon Availability Zone; in the Denver Local Zone, the hosted price rises to $0.28 per hour.</p> <p>Local Zones are limited in the types of server configurations that are available for deployment. For example, the Denver Local Zone does not have t3.large EC2 instances available for deployment, while all AZs do.</p> <h3>AWS Outposts</h3> <p>For extremely low-latency applications where a double-digit response time is unacceptable or where AWS Local Zones are unavailable, AWS customers have the option to <a href="https://www.techtarget.com/searchcloudcomputing/tip/What-you-need-to-know-to-deploy-an-AWS-hybrid-cloud-with-Outposts">purchase an AWS Outpost</a>.</p> <p>An AWS Outpost is a physical server costing an estimated $150,000 to $750,000 depending on the type of unit. It comes preconfigured with various AWS services such as RDS, S3, EKS and EC2. Since an AWS Outpost is a physical piece of hardware that sits inside a company's data center, the network latency for direct connections is negligible. Low, single-digit response times are possible.</p> <p>One of the primary use cases for AWS Outposts configured with RDS and S3 support is data sovereignty law compliance. When companies want to use Amazon services, but compliance regulations forbid offsite storage in the cloud, Outposts brings the AWS cloud into the client's on-premises data center.</p> <p>Configured with a high-speed, AWS Direct Connect linkage to a Local Zone or AZ, AWS Outposts are the lowest latency option. However, AWS Outposts does not support the same number of services as an AZ in an AWS Region.</p> </section> AWS offers its customers several options to minimize application latency. Let's look at the role AWS Local Zones can play in delivering your apps right on time. https://cdn.ttgtmedia.com/rms/onlineimages/location_g1202864734.jpg https://www.techtarget.com/searchcloudcomputing/tip/Are-AWS-Local-Zones-right-for-my-low-latency-app Tue, 27 Jun 2023 14:57:00 GMT <h3>What is Amazon CloudWatch?</h3> <p>Amazon CloudWatch is a component of Amazon Web Services that provides monitoring for AWS resources and the customer applications running on the&nbsp;Amazon infrastructure.</p> <p>CloudWatch enables <a href="https://www.techtarget.com/whatis/definition/real-time-monitoring">real-time monitoring</a> of AWS resources such as Amazon Elastic Compute Cloud (<a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Compute-Cloud-Amazon-EC2">EC2</a>) instances, Amazon Elastic Block Store (EBS)&nbsp;volumes, Elastic Load Balancing and&nbsp;Amazon Relational Database Service (<a href="https://www.techtarget.com/searchaws/definition/Amazon-Relational-Database-Service-RDS">RDS</a>) instances. The application automatically collects and provides metrics for&nbsp;CPU&nbsp;utilization,&nbsp;<a href="https://www.techtarget.com/whatis/definition/latency">latency</a> and request counts. Users can also stipulate additional metrics to be monitored, such as memory usage, transaction volumes or error rates.</p> <p>Users can access CloudWatch functions through an&nbsp;application programming interface (<a href="https://www.techtarget.com/searchapparchitecture/definition/application-program-interface-API">API</a>), command-line tools, one of the AWS&nbsp;software development kits or the AWS Management Console. The CloudWatch interface provides current statistics that users can view in graph format. Users can set notification alarms to be sent when something being monitored surpasses a specified threshold. The app can also detect and shut down unused or underused EC2 instances.</p> <p>Amazon CloudWatch is designed for AWS users, including DevOps engineers, IT managers, cloud developers and site reliability engineers.</p> <h3>CloudWatch features</h3> <p>CloudWatch enables users to collect and view monitoring data for AWS infrastructures in a single platform. CloudWatch includes specific features for data collection, monitoring, automated actions, analysis and compliance and security.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/security-4_steps_toward_cloud_sec-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/security-4_steps_toward_cloud_sec-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/security-4_steps_toward_cloud_sec-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/security-4_steps_toward_cloud_sec-f.png 1280w" alt="chart of how Amazon CloudWatch works " height="332" width="558"> <figcaption> <i class="icon pictures" data-icon="z"></i>What does Amazon CloudWatch do? </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <ul class="default-list"> <li><strong>CloudWatch Logs.</strong> This service enables users to collect and store logs for vended services for customers, logs for specific AWS services such as AWS CloudTrail, AWS Lambda, Amazon API Gateway, Amazon Simple Notification Service, or for proprietary applications and on-premises resources. CloudWatch Logs Insights can provide quick queries and visualization of log data.</li> <li><strong>Metrics collection.</strong> Users can collect default metrics from more than 70 distributed AWS applications and view them in one place. They also can collect metrics and customize logs from their own applications or on-premises resources.</li> <li><strong>Container Insights.</strong> This feature collects, aggregates and monitors metrics and logs for containerized applications and microservices. It can also <a href="https://www.techtarget.com/searchaws/tip/Use-CloudWatch-Container-Insights-to-troubleshoot-ECS-EKS">troubleshoot Amazon Elastic Kubernetes Service</a> and Amazon Container Orchestration Service.&nbsp;</li> <li><strong>CloudWatch Lambda Insights.</strong> This service collects, aggregates and monitors <a href="https://www.techtarget.com/searchaws/definition/AWS-Lambda-Amazon-Web-Services-Lambda">AWS Lambda</a> logs and performance metrics from each container, including CPU, memory and disk information.&nbsp; &nbsp;</li> <li><strong>Contributor Insights.</strong> This feature provides a view of the top contributors influencing system performance, such as API calls, applications or customer accounts.</li> </ul> <ul class="default-list"> <li><strong>Unified view. </strong>This feature enables users to create dashboard views for selected applications, graphs and other visualized cloud data.</li> <li><strong>Composite alarms. </strong>This function unifies alarms for different issues affected by the same application into a single notification. This can help <a href="https://www.techtarget.com/searchitoperations/definition/root-cause-analysis">root-cause diagnosis</a>.</li> <li><strong>High resolution alarms. </strong>Users can set thresholds for specific metrics that trigger alarm actions, such as shutting down unused instances.</li> <li><strong>Correlation. </strong>CloudWatch can correlate specific patterns in logs with metrics to diagnose a root cause.</li> <li><strong>Application Insights for .NET and SQL Server. </strong>This feature provides easy monitoring for .NET and SQL Server applications, with automated dashboards and smart metrics.</li> <li><strong>Anomaly Detection. </strong><a href="https://www.techtarget.com/searchenterpriseai/definition/machine-learning-ML">Machine learning</a> algorithms can detect abnormal activity in AWS systems.</li> <li><strong>ServiceLens. </strong>This service monitors the performance, health and availability of applications and dependencies to reduce bottlenecks, recognize affected users and diagnose root causes.</li> <li><strong>Synthetics. </strong>This facility monitors application endpoints and alerts the user to errors and abnormal infrastructure issues.</li> <li><strong>Metric Streams. </strong>This feature enables users to create near real-time metric streams to other applications, such as Amazon S3, or share them with third-party service providers.</li> <li><strong>Auto Scaling. </strong>This feature automates capacity and resource planning.</li> <li><strong>CloudWatch Events.</strong> This service provides a near real-time stream of system events and automates responses to operational changes.</li> <li><strong>Log analytics. </strong>Advanced analytics are available for the information in CloudWatch Logs, without provisioning additional servers or the need for extra software. Queries can be exported to dashboards.</li> <li><strong>Integration with AWS Identity and Access Management. </strong>This facility provides a management console to control which users and applications have access to CloudWatch data and resources.</li> </ul> <h3>CloudWatch vs. CloudTrail</h3> <p>CloudWatch and <a href="https://searchcloudsecurity.techtarget.com/definition/AWS-CloudTrail">CloudTrail</a> are both monitoring services for AWS resources and applications.</p> <p>AWS CloudWatch is a service that monitors system performance for AWS applications and resources, and AWS CloudTrail is a web service that monitors the activity within the AWS environment through tracking API calls.</p> <p>CloudTrail provides a detailed log of all actions in the AWS system, and helps users track user activity and changes, monitoring the trail of activity, hence the name. AWS CloudTrail provides information on the who, what, where and when of activity in the AWS account and environment.</p> <h3>Benefits of CloudWatch</h3> <p>CloudWatch offers several benefits for organizations using AWS resources and applications. These are related to the information that CloudWatch can provide, as well as its intuitive interface, and include the following.</p> <ul class="default-list"> <li>offers ease of use for basic functions;</li> <li>provides visibility for all AWS monitoring data on one platform;</li> <li>effectively collects <a href="https://www.techtarget.com/searchaws/tip/Master-Amazon-CloudWatch-metrics-to-track-AWS-use">metrics for AWS environments</a>;</li> <li>increases and optimizes operational performance of AWS and on-premises resources;</li> <li>provides insights into correlations and other relationships in system performance;</li> <li>provides stability and reliability; and</li> <li>offers integration with other AWS resources.</li> </ul> <h3>Challenges of CloudWatch</h3> <p>related to the use of AWS CloudWatch include the following:</p> <ul class="default-list"> <li>costs more than most <a href="https://www.techtarget.com/searchaws/tip/CloudWatch-Logs-Insights-wont-replace-third-party-tools-yet">third-party monitoring and log</a> tools;</li> <li>many standard AWS metrics cannot be seen in units smaller than one-minute intervals;</li> <li>advanced integration is mostly limited to AWS resources; and</li> <li>more advanced use of CloudWatch can have a high learning curve.</li> </ul> <h3>Amazon CloudWatch pricing</h3> <p>CloudWatch is available in free and paid tiers. The free tier includes limited monitoring metrics, API requests, dashboards, alarms, log data, events, contributor insights and <a href="https://www.techtarget.com/whatis/definition/canary-canary-testing">canary runs</a>.</p> <p>Like other cloud services, the paid tier follows a pay-as-you-go pricing structure that is billed monthly. Pricing is based on the quantity of metrics, APIs and metric streams used.</p> <p><em>See the </em><a href="https://aws.amazon.com/cloudwatch/pricing/?loc=ft#Free_tier" target="_blank" rel="noopener"><em>AWS CloudWatch pricing page</em></a><em> for a comprehensive breakdown with a pricing calculator and pricing examples.</em></p> <h3>Amazon CloudWatch use cases</h3> <p>CloudWatch collects data for operational and monitoring purposes with a unified view and can deploy automated responses to monitored metrics reaching a specified threshold. More broadly, CloudWatch is used for the following tasks:</p> <ul class="default-list"> <li>to help fix operational issues and optimize performance using log analytics;</li> <li>monitor AWS applications in the cloud or on premises;</li> <li>monitor and troubleshoot the AWS infrastructure; and</li> <li>optimize system resources.</li> </ul> <p><em>For more information on metrics in cloud monitoring, </em><a href="https://www.techtarget.com/searchcloudcomputing/feature/Metrics-that-matter-in-cloud-application-monitoring"><em>read here</em></a><em>. </em></p> <p><em>Learn how AWS </em><a href="https://www.techtarget.com/searchaws/tip/Compare-CloudWatch-vs-Datadog-and-New-Relic-for-AWS-monitoring"><em>CloudWatch compares</em></a><em> to other cloud monitoring services.</em></p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/Xg4yz33vvvU?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> <p></p> Amazon CloudWatch is a component of Amazon Web Services that provides monitoring for AWS resources and the customer applications running on the Amazon infrastructure. https://cdn.ttgtmedia.com/visuals/digdeeper/5.jpg https://www.techtarget.com/searchaws/definition/CloudWatch Mon, 17 May 2021 00:00:00 GMT <p>More and more developers run Docker-based applications in the cloud, and Amazon's container services simplify the launch and management of those applications on AWS. However, the offerings do have drawbacks when it comes to the limited set of available <a href="https://www.techtarget.com/searchcloudcomputing/tip/Use-these-CloudWatch-features-to-improve-AWS-app-monitoring">CloudWatch</a> metrics.</p> <p>For example, Amazon Elastic Container Service (<a href="https://www.techtarget.com/whatis/definition/cluster">ECS</a>) publishes metrics at the cluster level, such as CPU and memory utilization and reservation, as well as similar metrics at the service level. While this information can help determine the overall health of ECS infrastructure, it's not sufficient to effectively prevent and remediate all AWS container issues.</p> <p>CloudWatch Container Insights, a feature within AWS' native monitoring and management service, provides a more detailed set of metrics at the cluster, service and task level compared to default ECS metrics. Developers can use Container Insights to monitor the number of desired, pending or running tasks for a particular service. This monitoring capability can detect issues related to launching tasks -- when tasks are stuck in pending state and cannot be launched, or when a task is launched too frequently, which could be a sign of an unhealthy container.</p> <p>Developers can also use Container Insights to monitor CPU and memory utilization at the ECS task level, which offers a more granular view of system health. CloudWatch Container Insights can examine received and transmitted bytes at the cluster, service and task definition level, which can help developers discover and troubleshoot issues in a way that isn't possible with default ECS metrics.</p> <p>When it comes to Amazon Elastic Kubernetes Service (EKS) clusters, CloudWatch Container Insights is particularly useful, since EKS doesn't automatically publish metrics to CloudWatch. Here, Container Insights provides visibility at the <a href="https://www.techtarget.com/searchitoperations/tip/Kubernetes-Pods-and-Nodes-map-out-container-relationships-for-production">cluster, node and pod</a> level. Users can access metrics that capture the total number of worker nodes in the cluster and the total percentage of CPU units used on cluster nodes, among others. With these metrics, users can closely monitor the overall Kubernetes infrastructure health and react to potential issues.</p> <p>Developers can trigger <a href="https://www.techtarget.com/searchaws/news/252433485/AWS-Auto-Scaling-simplifies-management-across-services">AWS Auto Scaling</a> actions based on resource utilization at the node level and set alarms for when pods are unhealthy. They can also better troubleshoot their AWS container infrastructure by visualizing detailed metrics in a CloudWatch dashboard. These metrics can also be enabled in <a href="https://www.techtarget.com/searchaws/news/252433485/AWS-Auto-Scaling-simplifies-management-across-services">self-managed Kubernetes</a> clusters running on EC2 instances.</p> <section class="section main-article-chapter" data-menu-title="Set up CloudWatch Container Insights"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Set up CloudWatch Container Insights</h2> <p>CloudWatch Container Insights isn't enabled by default, so developers have to explicitly configure it. The setup process is relatively simple for ECS clusters and can be completed with one step using either the ECS console for new clusters or the AWS Command Line Interface (CLI) for existing clusters.</p> <p>For EKS or self-managed Kubernetes clusters, the process isn't as simple, as it requires users to install the CloudWatch agent as well as data collector <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-logs.html" target="_blank" rel="noopener">FluentD</a>. Users will have to configure Identity and Access Management permissions, download custom files, use the kubectl CLI and undertake a few other steps.</p> <p>For the time being, developers can't enable the feature using CloudFormation for either ECS or EKS, which is a limitation that AWS could improve in the future.</p> </section> <section class="section main-article-chapter" data-menu-title="How to access and use CloudWatch Container Insights metrics"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How to access and use CloudWatch Container Insights metrics</h2> <p>CloudWatch automatically builds a detailed dashboard with Container Insights metrics. Enable and view the dashboard in the CloudWatch console, or access Container Insights metrics using the CloudWatch API or CLI, as Container Insights metrics are published as custom CloudWatch metrics. You can also use these metrics to trigger CloudWatch alarms and add them to custom CloudWatch Dashboards.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineImages/cloudwatchdash.jpg"> <img data-src="https://www.techtarget.com/rms/onlineImages/cloudwatchdash_mobile.jpg" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineImages/cloudwatchdash_mobile.jpg 960w,https://www.techtarget.com/rms/onlineImages/cloudwatchdash.jpg 1280w" alt="Container Insights Dashboard" height="289" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>The CloudWatchContainer Insights dashboard enables you to track all your relevant metrics in one hub. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p>CloudWatch Container Insights also creates entries in <a href="https://www.techtarget.com/searchaws/answer/How-do-CloudWatch-logging-features-track-resources">CloudWatch Logs</a>, which enables users to submit their own container-related queries using CloudWatch Logs Insights. This supports more detailed analysis of log entries and deeper visibility into individual metric events, which is useful during troubleshooting activities.</p> </section> <section class="section main-article-chapter" data-menu-title="CloudWatch Container Insights pricing"> <h2 class="section-title"><i class="icon" data-icon="1"></i>CloudWatch Container Insights pricing</h2> <p>As previously stated, Container Insights publishes metric data as custom CloudWatch metrics, which start at $0.30 per metric, at time of publication. That rate decreases after an account's first 10,000 metrics. A single ECS cluster with one service deployed in it and one task running in the service would publish 16 custom metrics. This would cost $4.80 per month in the U.S. East-1 region, assuming your account hasn't passed 10,000 custom metrics yet. For a single EKS cluster with one namespace, node and pod, there would be 25 custom metrics published to CloudWatch, resulting in a charge of $7.50 per month.</p> <p>There's also some associated cost to ingest and store data in CloudWatch Logs -- $0.50 per gigabyte ingested and $0.03 per gigabyte per month of data stored. In most situations, CloudWatch Logs costs will likely be low, given the volume of data users send to it.</p> </section> Ensure container-based apps run properly in AWS with CloudWatch Container Insights. Discover how to set up and use this feature for detailed assessments of ECS or EKS workloads. https://cdn.ttgtmedia.com/visuals/searchCloudComputing/applications/cloudcomputing_article_001.jpg https://www.techtarget.com/searchaws/tip/Use-CloudWatch-Container-Insights-to-troubleshoot-ECS-EKS Thu, 31 Oct 2019 13:01:00 GMT <p>In April 2012,&nbsp;<a href="https://www.techtarget.com/searchaws/definition/Amazon-Web-Services">Amazon Web Services</a>&nbsp;launched AWS Marketplace, a one-stop spot for finding software that runs on AWS. Today, AWS Marketplace contains 37 categories that include software infrastructure, business software, desktop software and developer tools.</p> <p>The online store is among&nbsp;AWS' fastest-growing services, regularly adding new technology partners that seek to fill service gaps or otherwise appeal to AWS' many customers.</p> <p>But how does AWS Marketplace work? And what can you expect to find there? These frequently asked questions and answers shed some light on the expansive AWS Marketplace.</p> <h3>What is the AWS Marketplace?</h3> <p>Amazon describes AWS Marketplace as "an online store that helps customers find, buy and immediately start using the software and services they need to build products and run their businesses." It also serves as a new sales channel for software vendors.</p> <h3>What drove AWS to create its online store?</h3> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> Software sold in AWS Marketplace is available to run only on Amazon EC2; it's not available for download. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> <p>Amazon said customers have been asking for faster, better ways to find and deploy&nbsp;software running on AWS. In a blog post, Amazon CTO Werner Vogels wrote, "The way businesses are buying applications is changing. There is a new generation of leaders that have very different expectations about how they can select the products and tools they need to be successful." Specifically, he said, Amazon created AWS Marketplace to meet customer demands for choosing software "in a way that was completely in line with the 'cloud experience:' no software to install, no sales cycle, no procurement delays and [with] a selection of licensing models to choose from."</p> <h3>Who can shop at the AWS Marketplace?</h3> <p>Any customer with an AWS account can shop in AWS Marketplace. Customers should be familiar with <a href="https://www.techtarget.com/searchaws/definition/Amazon-Elastic-Compute-Cloud-Amazon-EC2">Amazon Elastic Compute Cloud</a>&nbsp;(EC2) instances before purchasing vendor software.</p> <h3>What types of products are sold on AWS Marketplace?</h3> <p>The store offers commercial&nbsp;software products targeted to companies and developers,&nbsp;as well as open source software. The four major product categories include:</p> <ol start="1" type="1" class="default-list"> <li>Software infrastructure, which includes OSes, security, migration and big data tools;</li> <li>Developer tools, such as issue and bug tracking, monitoring, source control and testing;</li> <li>Business software in categories ranging from business intelligence to collaboration to e-commerce to storage and recovery; and</li> <li>Desktop software, which includes Microsoft Windows desktop applications for accounts payable and billing, media and encoding, project management and other Windows apps.</li> </ol> <p>Software sold in AWS Marketplace is available to run only on Amazon EC2; it's not available for download.</p> <h3>How does AWS Marketplace work?</h3> <p>Customers pick the preconfigured software they want to buy and launch it immediately using AWS' 1-Click Deployment, which provides a similar experience to the 1-Click Ordering option available to consumers on Amazon.com. (If customers prefer, they can skip the one-click installation and develop the software using their own tools.) Their purchases are available either via <a href="https://www.techtarget.com/searchcloudcomputing/definition/Software-as-a-Service">SaaS</a> or as an Amazon Machine Image (AMI).</p> <h3>What is an AMI?</h3> <p>AWS describes an <a href="https://www.techtarget.com/searchaws/definition/Amazon-Machine-Image-AMI">AMI</a> as a template with a software configuration for the customer's server -- for instance, an OS or an application server. In the case of AWS Marketplace, an AMI is a template carrying the information needed to boot an Amazon EC2&nbsp;<a href="https://www.techtarget.com/searchaws/definition/Amazon-EC2-instances">instance</a>&nbsp;with the customer's software.</p> <h3>How does AWS Marketplace customer billing work?</h3> <p>AWS Marketplace offers a pay-as-you-go pricing model. For paid software, customers pay only for what they use, either by the hour or by the month, scaling their usage as needed. Charges from AWS Marketplace are integrated into each customer's existing AWS bill.</p> <h3>Who sells software in AWS Marketplace?</h3> <p>Vendors are members of the AWS Partner Network, the AWS global program for its technology and consulting partners. They include&nbsp;<a href="https://www.techtarget.com/searchitchannel/definition/ISV">independent software vendors</a>,&nbsp;<a href="https://www.techtarget.com/searchitchannel/definition/VAR">value-added resellers</a>&nbsp;and&nbsp;<a href="https://www.techtarget.com/searchitchannel/definition/systems-integrator">systems integrators</a>.</p> <h3>What companies currently use AWS Marketplace?</h3> <p>Participating SaaS vendors include CA, Check Point, IBM, Microsoft, <a href="https://www.ogf.org/OGF24/materials/1448/Ent02+-+SaaS+on+the+Grid.pdf" target="_blank">Red Hat</a> and SAP, among many other large and small sellers. Open source offerings include Drupal, MediaWiki and WordPress.</p> <h3>How can I become an AWS Marketplace seller?</h3> <div class="imagecaption alignRight"> <iframe width="220" height="182" src="https://www.youtube.com/embed/4YPH0_Vis84?ecver=1" frameborder="0" allowfullscreen></iframe> <br> <i>A tutorial of AWS Marketplace with WordPress</i> </div> <p>According to Amazon, vendors completing a seller registration must sell publicly available software that fits into one of the store's supported categories. Vendors must also have "a strong track record of customer satisfaction" and provide tax information for paid AMI products. They must demonstrate strong, clear business operations and ethical practices. They must place high priority on keeping their products up-to-date and virus-free. And they must verify that the AWS cloud hosts and runs their software on EC2 instances.</p> <h3>How does AWS Marketplace handle vendor payments?</h3> <p>Amazon handles billing for all software purchases and manages payouts to participating vendors. When an AWS customer purchases vendor software, the cloud provider adds the cost to the customer's AWS bill. Sellers can apply several different pricing models to their software, including bring your own license, free, hourly, monthly and usage-based billing. AWS might charge seller fees depending on the pricing model a vendor chooses. The AWS Marketplace seller guide contains a full list of vendor guidelines and recommendations.</p> AWS Marketplace serves Amazon Web Services customers and vendor partners alike. So, how does it work, and what types of software are available? https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/feature/FAQ-How-the-AWS-Marketplace-serves-software-buyers-and-sellers-alike Tue, 25 Jul 2017 15:00:42 GMT <p>AWS AppStream is a new Amazon service that streams Windows applications in the cloud, extending the OS to a variety of computers and mobile devices. Today, the most practical use cases for the service are providing brief game demos and longer-term support for high-value applications, like CAD and medical imaging. In the case of CAD applications, AppStream allows an engineer to make quick updates in the field from a tablet. In medical imaging applications, AWS AppStream makes it easier for multiple medical experts to collaborate remotely.</p> <section class="section main-article-chapter" data-menu-title="Start streaming the client application"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Start streaming the client application</h2> <p>The first consideration for <a href="https://www.techtarget.com/searchaws/tips">AWS AppStream</a> adoption lies in identifying the business and revenue model behind AppStream apps. Today, it costs 83 cents per hour to deploy these apps in the US East region. This might be an acceptable cost to provide gamers an instant demo of a new game, but it would not make sense for sustained game play in many cases. It may also be an acceptable cost for high-end CAD as a service offerings that are provisioned to allow engineers to make changes in the field on a low-end tablet.</p> <p>The client application works in a dedicated AppStream app that runs locally on each device. The local app decodes audio and video sent from Amazon Web Services (AWS) and collects user input from a keyboard, mouse and touch screen. Users can configure the local app to gather additional input, like GPS data, using the AppStream <a href="https://www.techtarget.com/whatis/definition/software-developers-kit-SDK">software development kit</a>. Client applications are available for iOS, Android, Chrome, Windows and Mac operating systems. The application must be able to run on Windows Server 2008 or later. If the application has other dependencies, such as the .NET framework, these need to be included as part of the application installer.</p> </section> <section class="section main-article-chapter" data-menu-title="Bringing high-end performance to low-end devices"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Bringing high-end performance to low-end devices</h2> <p>The underlying notion of desktop virtualization has been around for years. In this traditional model, a Windows desktop instance is provisioned in the cloud. This approach makes it easier to spin up and provide an entire Windows environment with the appropriate configurations and application collections. AppStream takes this to a more granular level by allowing IT to provision individual applications or collections. It also includes features for integration with custom clients, subscriptions, identity and storage.</p> <p>By streaming applications, organizations can better distribute computational loads for CPU- and GPU-intensive applications. For example, applications like <a href="https://www.techtarget.com/searchvirtualdesktop/tip/5-tips-to-optimize-CAD-application-performance-for-VDI">CAD</a>, 3D modeling, video editing and medical imaging can move the heavy processing to the cloud. The client application can then render the results using the standard video decoder chips already common on tablets and phones. In essence, any application that can benefit from additional CPU, GPU, RAM or storage requirements can benefit from AppStream.</p> </section> <section class="section main-article-chapter" data-menu-title="Understand network effects"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Understand network effects</h2> <p>An important consideration when moving Windows applications to the cloud is <a href="https://www.techtarget.com/searchnetworking/tip/How-to-improve-network-latency-in-3-steps">latency</a>, which creates a perception of delay between user action and its effect on the application. This is less important for applications like CAD annotation that can tolerate 100-millisecond delays. But gaming experiences might suffer with delays as little as 20 milliseconds. Latency is affected by the physical distance a data packet has to travel to the cloud, the number of routers through which it has to pass and the amount of competing Internet traffic. Loses incurred by wireless networks can also exacerbate latency issues.</p> <p>Another important consideration is that AppStream requires a live Internet connection. This could be a problem for mobile users outside the coverage areas of cell phone towers. However, it is possible to divide hosting duties between native resources and AppStream to take advantage of better processing. In these cases, users can continue to work when the Internet connection goes down, albeit with less performance. However, these hybrid apps require more development work for each supported platform.</p> <p>AppStream requires a minimum of 3 megabytes per second (MBps) for streaming applications. This is probably not a big deal for individual users with a cable or DSL connection, but it is when dozens of workers are trying to simultaneously run AppStream applications. It could also quickly push mobile workers over their wireless network data plan allotments. Applications will continue to function at lower bit-rates -- down to 553 KBps -- and at lower quality.</p> </section> <section class="section main-article-chapter" data-menu-title="Steps to getting started"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Steps to getting started</h2> <p>AppStream supports an interactive deployment wizard that converts a Windows application installer into an AppStream instance. Initial application performance may appear slow during this process, as it relies on the legacy Windows <a href="https://www.techtarget.com/searchenterprisedesktop/definition/Remote-Desktop-Protocol-RDP">Remote Desktop Protocol</a>. Once the application is running in the cloud, it will use the native AWS AppStream STX protocol, which has been optimized to improve application response to latency.</p> <p>Next, secure the AppStream application by setting up an entitlement service to authenticate and authorize users. To do this, compare users to subscribers in a database, use an external log-in service or authenticate all clients. The entitlement service sends the client a URL to connect to the live application.</p> <p>AppStream data is lost when the client session ends. If data needs to be shared between AppStream apps or persist between client sessions, set up some form of persistent storage. The AppStream console includes features for integrating and preserving session data using <a href="https://www.techtarget.com/iotagenda/definition/Internet-of-Things-IoT">Simple Storage Service</a>, <a href="https://www.techtarget.com/searchaws/tip/Perform-an-Amazon-RDS-failover-with-minimal-downtime">Relational Database Service</a> or DynamoDB.</p> <p>To track user experience and resource utilization, users can capture log data from AppStream sessions. This information can also be used to identify bottlenecks to application performance.</p> </section> While a valuable service for AWS users looking to connect to different types of devices, AppStream has network and security limitations IT professionals should consider. https://cdn.ttgtmedia.com/visuals/searchAWS/infrastructure/aws_article_008.jpg https://www.techtarget.com/searchaws/tip/What-to-know-before-going-all-in-with-AWS-AppStream Mon, 24 Aug 2015 16:15:38 GMT <p>There are a few common issues all developers encounter when running applications in Amazon Elastic Beanstalk, especially if they are unfamiliar with Elastic Load Balancing and Auto Scaling Groups. This tip covers a few best practices for working with Elastic Load Balancing to ensure AWS applications run as expected.</p> <section class="section main-article-chapter" data-menu-title="Configure Elastic Load Balancing and security groups"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Configure Elastic Load Balancing and security groups</b></h2> <p>If you're automatically redirecting HTTP traffic to HTTPs, enable HTTPS proxying on Elastic load balancing (ELB) and upload an SSL certificate. In addition, open up HTTPS on the security group for the outside world and ensure HTTP is also open to the outside world.</p> <p>If your load balancer can't connect to the AWS instance, make sure the security group that hosts the load balancer can access the port in which your service is running on application servers. Application servers run in a different security group than the ELB, and the load balancer must have access to them.</p> </section> <section class="section main-article-chapter" data-menu-title="Set Auto Scaling Group to use ELB health checks"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Set Auto Scaling Group to use ELB health checks</b></h2> <p>By default, the Auto Scaling Group (ASG) will check that instances are running, which could indicate that services on the AWS instance are working. While using the ELB health check provides a few false-error conditions, such as when a server is recovering from high load, it can ensure your service remains responsive. Play it safe; have the ASG health check use the ELB health check.</p> <p>In addition, be sure there's enough time specified on the ASG's <i>Health Check Grace Period</i> or the time the system waits after a server is spawned before it assumes it didn't work. I've had issues where this was so low the ASG continually spawned and then killed instances because they took too long to spin up.</p> <p>Setting this up is as simple as defining an .ebextension in the repository. This works only on the first time you create the environment, so be sure to add it<i> </i>before you spin up the environment. You can also add something like the following under .ebextensions/asg.config:</p> <p>Resources:</p> <p>AWSEBAutoScalingGroup:</p> <p>Type: AWS::AutoScaling::AutoScalingGroup</p> <p>Properties:</p> <p>HealthCheckGracePeriod: 600</p> <p>HealthCheckType: ELB</p> </section> <section class="section main-article-chapter" data-menu-title="Setting up Virtual Private Cloud routing"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Setting up Virtual Private Cloud routing</b></h2> <p>AWS instances need access to the outside world and to Amazon. One common issue is that developers accidentally set up a subnet group that doesn't have access to the Internet and launch the environment there. If you've blocked access to the outside world, servers won't be able to download the software they need to spin up and, most likely, won't be able to contact Elastic Beanstalk to update their own status. Check the VPC subnet configuration to be sure servers can access the outside world. And check that routes have a path to <i>0.0.0.0/0</i> that is not in a black hole.</p> </section> <section class="section main-article-chapter" data-menu-title="Planning subnets for ELB and application servers"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Planning subnets for ELB and application servers</b></h2> <p>If you set up a new VPC and try to save time by only creating only one subnet, it won't work. While you can apply the same route to Elastic Load Balancing and application servers, they cannot be in the same subnet because this causes countless problems.</p> <p>App servers don’t need public IP addresses, but load balancers do. App servers can proxy Internet traffic through NAT instances; load balancers can't. Place the load balancer in the same availability zones (AZ) as application servers and always set up at least two AZs. That means the typical setup is four subnets split across two AZs. Reserve two subnets for the load balancer and two for app servers. And be sure the subnets are large enough to support all the application servers you need. Keep more available in case ELB needs to scale.</p> </section> <section class="section main-article-chapter" data-menu-title="Prepare for load balancer changes"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Prepare for load balancer changes</b></h2> <p>If you make major changes or update the environment, the load balancer may change. This is important because you can't make a CNAME to a root domain, such as Amazon.com, but you can make an alias to Elastic Load Balancing. To determine what the load balancer is for an environment, check to see what instances are associated to it.</p> <p>If you use an alias via Route 53 to point a root domain to the load balancer, check it every time you make a major change, such as replacing the environment. Keep in mind: DNS takes a few minutes to propagate, so be sure to spin up the replacement, make the change to Route 53 and wait at least a day before terminating the old environment.</p> </section> Improperly configuring Elastic Load Balancing and Auto Scaling Groups in Elastic Beanstalk can harm AWS app performance. Avoid these setup mistakes. https://cdn.ttgtmedia.com/visuals/searchAWS/cloud_development/aws_article_005.jpg https://www.techtarget.com/searchaws/tip/Elastic-Load-Balancing-tricks-for-AWS-applications Fri, 17 Apr 2015 16:49:31 GMT <p><a href="https://www.techtarget.com/iotagenda/definition/Internet-of-Things-IoT">Amazon Simple Storage Service</a> (S3) is mostly known for its object-based storage for storing data, but S3 bucket policy can help AWS save time in setting and managing complex access rights for Amazon S3 resources. This tip guides developers through basic S3 practices. Note that S3 is usually used with Amazon <a href="https://www.techtarget.com/searchcio/feature/IAM-driven-biometrics-in-security-requires-adjustments">Identity and Access Management</a> (IAM) policy, which is used to set up, capture and record user identities and permissions. I'll cover IAM in a future tip.</p> <p>You need an S3 bucket policy to access a file you've uploaded. If you don't have it in place, you will get an error message that you are denied access.</p> <table border="1" cellspacing="0" cellpadding="3"> <tbody> <tr> <td width="664" valign="top">This XML file does not appear to have any style information associated with it. The document tree is shown below</td> </tr> </tbody> </table> <p>&nbsp;</p> <table border="1" cellspacing="0" cellpadding="3"> <tbody> <tr> <td width="665" valign="top"> <p><code>1 -&lt;error&gt;</code></p> <p><code>2 &lt;code&gt;<b>AccessDenied</b>&lt;/code&gt;</code></p> <p><code>3 &lt;message&gt;<b>Access Denied</b>&lt;/message&gt;</code></p> <p><code>4 &lt;requestid&gt;<b>F7A33F55E19C8BFA</b>&lt;/requestid&gt;</code></p> <p><code>5 -&lt;hostid&gt;</code></p> <p><code>6 <b>UHwvZfsh+B9IczJIyrBQOKG1+JRVsybONoS8+pwo1DZSvscmdb9OIsnZw</b>&lt;/hostid&gt;</code></p> <p><code>7 &lt;/error&gt;</code></p> </td> </tr> </tbody> </table> <p>This message pops up after you click on the linked file name in the Properties window. To see this window, click on the Properties tab on your right of the All Buckets window.</p> <p>To fix the problem, create an S3 bucket policy. Use the <a href="http://awspolicygen.s3.amazonaws.com/policygen.html" target="_blank">AWS Policy Generator</a> to generate a script that allows you to access your file.</p> <p>I have two script examples to show you how to set permissions. The first example is a simple script to permit anyone to access my files. The second example is a bit longer script to allow all users, except one, to upload their files to my bucket.</p> <p>For the first example, here are the three steps to follow:&nbsp;</p> <section class="section main-article-chapter" data-menu-title="Step 1. Select policy type"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Step 1. Select policy type</b></h2> <p>Choose<b>S3 bucket policy.</b></p> <p>This policy is one of the four types of policies you can create. The other three are:</p> <ul class="indentfix"> <li class="indentfix"><a href="http://docs.amazonwebservices.com/IAM/latest/UserGuide/PermissionsAndPolicies.html" target="_blank">IAM Policy</a></li> <li class="indentfix"><a href="http://docs.amazonwebservices.com/sns/latest/gsg/AccessPolicyLanguage.html" target="_blank">SNS Topic Policy</a></li> <li class="indentfix"><a href="http://docs.amazonwebservices.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/AccessPolicyLanguage.html" target="_blank">SQS Queue Policy</a></li> </ul> </section> <section class="section main-article-chapter" data-menu-title="Step 2. Add statements"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Step 2. Add statements</b></h2> <p>A statement is the formal description of a single permission. There are five elements in a statement. They are:</p> <ul class="indentfix"> <li class="indentfix">Effect element</li> <li class="indentfix">Principal element</li> <li class="indentfix">AWS service element</li> <li class="indentfix">Actions element</li> <li class="indentfix">Amazon Resource Name (ARN) element</li> </ul> <p>For the Effect element, choose <b>Allow</b> to permit users to read a file.</p> <p>For the Principal element, add <b>*</b> to the input box. It is a wildcard to allow anonymous users to read the file.</p> <p>For the AWS Service element, the generator automatically inserts <b>Amazon S3</b> in the input box.</p> <p>For the Actions element, check off <b>Get Object. </b>This is the only action you need to read a file. You don't need any other actions in the list.</p> <p>For the Amazon Resource Name element, set it to the following format.</p> <p align="center">arn:aws:s3:::&lt;bucket_name&gt;/&lt;key_name&gt;</p> <p>Choose a bucket name (e.g., bucket01) that you've created. Set the key name to *. Enter the following in the input box:</p> <p align="center">arn:aws:s3:::bucket01/*</p> <p>Skip conditions about the statement such as date, location, version ID, maximum keys and other restrictions.</p> <p>Click the <b>Add Statement</b> button when you are ready.</p> <p>In an instant, you will see the result of what elements you've added to the statement.</p> <p>Once you are satisfied with the result, proceed to the next step.</p> </section> <section class="section main-article-chapter" data-menu-title="Step 3. Create bucket policy"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Step 3. Create bucket policy</b></h2> <p>Press the <b>Generate Policy</b> button. This will generate the policy you will need to add to your bucket. The policy is written in the <a href="http://docs.amazonwebservices.com/IAM/latest/UserGuide/AccessPolicyLanguage.html" target="_blank">Access Policy Language.</a></p> <table border="1" cellspacing="0" cellpadding="3"> <tbody> <tr> <td width="605" valign="top"> <p><code>{</code></p> <p><code>&nbsp;<b>"Id": "Policy1393570093893",</b></code></p> <p><code><b>&nbsp;"Statement": [</b></code></p> <p><code><b>&nbsp; {</b></code></p> <p><code><b>&nbsp;&nbsp; "Sid": "Stmt1393569661962",</b></code></p> <p><code><b>&nbsp;&nbsp; "Action": [</b></code></p> <p><code><b>&nbsp;&nbsp;&nbsp; "s3:GetObject"</b></code></p> <p><code><b>&nbsp;&nbsp; ],</b></code></p> <p><code><b>&nbsp;&nbsp; "Effect": "Allow",</b></code></p> <p><code><b>&nbsp;&nbsp; "Resource": "arn:aws:s3:::bucket01/*",</b></code></p> <p><code><b>&nbsp;&nbsp; "Principal": {</b></code></p> <p><code><b>&nbsp;&nbsp;&nbsp; "AWS": [</b></code></p> <p><code><b>&nbsp;&nbsp;&nbsp;&nbsp; "*"</b></code></p> <p><code><b>&nbsp;&nbsp;&nbsp; ]</b></code></p> <p><code><b>&nbsp;&nbsp; }</b></code></p> <p><code><b>&nbsp; }</b></code></p> <p><code><b>&nbsp;]</b></code></p> <p><code><b>}</b></code></p> </td> </tr> </tbody> </table> <p>If the script shows you've given a wrong bucket name, you can edit it. Once you are satisfied with the policy, copy it onto your clipboard. You will need to paste it on a policy editor.</p> <p>Here's how to get to the policy editor:</p> <ul class="indentfix"> <li class="indentfix">Open your bucket in the <a href="http://aws.amazon.com/console/" target="_blank">AWS Management Console</a></li> <li class="indentfix">Go to <b>Properties</b> and click <b>Permissions</b></li> <li class="indentfix"><b>Select Add bucket policy</b></li> </ul> <p>Once the editor opens, paste the policy. Then click <b>Save. </b>The editor provides a link to <b>Sample Bucket Policies.</b></p> <p>In the second S3 bucket policy example, the bucket owner with full control permits all accounts, except one, to upload files. The bucket owner sets a certain condition that the one account must meet before granting the permission to upload files. The PutObject operation is used to add a file to a bucket.</p> <table border="1" cellspacing="0" cellpadding="3"> <tbody> <tr> <td width="665" valign="top"> <p><code>{</code></p> <p><code>&nbsp;&nbsp;"Version":"2012-10-17",</code></p> <p><code>&nbsp;&nbsp;"Statement":[</code></p> <p><code>&nbsp;&nbsp; {</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Sid":"111",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Effect":"<b>Allow</b>",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Principal":{</code></p> <p><code>&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;"AWS":"<b>123456789</b>"</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;},</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Action":"<b>s3:PutObject</b>",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Resource":"<b>arn:aws:s3:::bucket01</b>/*"</code></p> <p><code>&nbsp;&nbsp; },</code></p> <p><code>&nbsp;&nbsp; {</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Sid":"112",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Effect":"<b>Deny</b>",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Principal":{</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "AWS":"<b>123456789</b>"</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;},</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Action":"<b>s3:PutObject</b>",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Resource":"<b>arn:aws:s3:::bucket01/*</b>",</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;"Condition":{</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "StringNotEquals":{</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;"<b>s3:x-amz-grant-full-control</b>":[</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; "<b>emailAddress=xyz@amazon.com</b>"</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;]</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }</code></p> <p><code>&nbsp;&nbsp;&nbsp; &nbsp;}</code></p> <p><code>&nbsp;&nbsp; }</code></p> <p><code>&nbsp;&nbsp;]</code></p> <p><code>}</code></p> </td> </tr> </tbody> </table> <p>This script starts off with the first statement allowing account 123456789 to upload a file in bucket01 only if the condition is not satisfied in the second statement. If account 123456789 does not email the bucket owner granting him the full control, then uploading the files is denied.</p> <p>In conclusion, you need a bucket policy to grant or deny accounts to read and upload files in your bucket. Amazon provides a reference on operations on <a href="http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketOps.html" target="_blank">buckets</a> and <a href="http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectOps.html" target="_blank">objects</a>.</p> </section> In this AWS Management Console series, learn five ways to create an S3 bucket policy. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/tip/AWS-Management-Console-Five-steps-to-create-an-S3-bucket-policy Fri, 18 Apr 2014 09:26:58 GMT <p>Simple Storage Service (S3) browser is a client interface for AWS developers using <a href="https://www.techtarget.com/iotagenda/definition/Internet-of-Things-IoT">Amazon S3 service</a>. Its user-friendly <a href="https://www.techtarget.com/whatis/definition/GUI">GUI</a> aids them in managing buckets of files of any size. The browser can be used as an interface for <a href="https://www.techtarget.com/searchaws/answer/Whats-the-difference-between-Amazon-S3-and-Amazon-CloudFront">Amazon CloudFront</a>.</p> <p>In a previous tip, I showed you how to create a bucket policy for a bucket that you own. If you are the owner of many buckets, you may wish to edit the original bucket policy for each bucket.</p> <p>To make life easier when editing the policies, meet <a href="https://www.techtarget.com/searchaws/tip/Process-large-data-sets-quickly-with-Amazon-Elastic-MapReduce">S3</a> Browser with its friendly GUI. If this is the first time you're using this browser after you <a href="http://s3browser.com/download.php" target="_blank">download</a> it, you must create a new account.</p> <p>You will be prompted for</p> <ul type="disc" class="default-list"> <li class="indentfix">Account name</li> <li class="indentfix">Access Key ID</li> <li class="indentfix">Secret Access Key</li> </ul> <p>The browser will give you the URL address where you can pick up your access keys that <a href="http://amazon.com">Amazon</a> generates for you. Put them in a safe place.</p> <p>You have the option to encrypt access keys with a password that you will need to get in the S3 Browser. When you're done, the browser icon will be placed on your home screen.</p> <p>If you already have created a bucket policy for a bucket, you may choose to copy and edit it for another bucket. Here are the steps you should take:</p> <p>To get to a policy you want to copy:</p> <ul type="disc" class="default-list"> <li class="indentfix">Highlight a bucket</li> <li class="indentfix">Click <b>Edit Bucket Policies</b> under the <b>Buckets</b> Tab</li> <li class="indentfix">Copy the policy in the editor</li> <li class="indentfix">Click <b>Apply bucket policies</b></li> <li class="indentfix">Close the editor</li> </ul> <p>To edit the copied policy:</p> <ul type="disc" class="default-list"> <li class="indentfix">Highlight the other bucket</li> <li class="indentfix">Click <b>Edit Bucket Policies </b></li> <li class="indentfix">Paste and make changes to the policy in the editor</li> <li class="indentfix">Click <b>Apply bucket policies</b></li> <li class="indentfix">Close the editor</li> </ul> <p>You should test the bucket policy to make sure it works as you intended.</p> <p>Another useful tool is the <b>Bucket Sharing Wizard. </b>You first create a new user and grant him permissions to your buckets and files. You click the button to take you to the next window.</p> <p>Amazon generates <b>Access Key Id </b>and <b>Secret Access Key. </b>Click the button to copy the keys for that user (note: The free version of S3 Browser doesn't provide this option).</p> <p>To edit the policy, click <b>Identity</b> <b>Access Management (IAM)</b> under <b>Tools</b>, then on the desired user in the left panel. The policy appears in the right panel.</p> <p>Amazon CloudFront is a good choice for distribution of frequently accessed static content like popular website images, videos, media files or software downloads.</p> <p>If this is the first time you're creating a CloudFront distribution, go to <b>CloudFront</b> in the <b>Bucket</b> tab. Choose a delivery method:</p> <ul type="disc" class="default-list"> <li class="indentfix">Download</li> <li class="indentfix">Streaming</li> <li class="indentfix">Live HTTP with Flash Media Server</li> </ul> <p>You can enable logging of your distributions.</p> <p>To edit, view, delete and create CloudFront distributions, choose <b>CloudFront Manager</b> under <b>Tools</b>.</p> <p>The S3 Browser can process very large amounts of files up to 5 TB. If the upload gets interrupted, you can resume uploading. To find out whether the uploading has been completed or not, go to <b>Tools</b>, and then choose <b>Uncompleted Multipart Uploads.</b></p> <p>To ensure the files are not corrupted during downloading or uploading, check the data integrity of these files by following these steps:</p> <ul type="disc" class="default-list"> <li class="indentfix">Go to <b>Options</b> under <b>Tools</b></li> <li class="indentfix">Choose <b>Data Integrity</b></li> <li class="indentfix">Check off the boxes to perform data integrity</li> <li class="indentfix">Choose the <b>Strict</b> mode, which treats the files as missing or invalid <a href="https://www.techtarget.com/searchcontentmanagement/tip/Consider-these-electronic-signature-best-practices">MD5 hashes</a> (Message-Digest algorithm 5)</li> </ul> <p>The above features are available in the free version of S3 Browser. Developers who want to run the application in commercial, business, government or military institutions must purchase a Pro version license.</p> <p>You can get one lifetime license for installing S3 Browser on a single machine for $29.95. You can transfer the license if you change your PC. The license is lifetime and includes one year of free upgrades and support. Amazon offers volume discount pricing.</p> <p>S3 Browser Pro allows you to upload and download multiple files at the same time. All files larger than 5 Mb can be uploaded via multiple parts in parallel. The free version allows up to two concurrent uploads and downloads.</p> <p>To adjust the number of concurrent uploads and downloads, try this:</p> <ul type="disc" class="default-list"> <li class="indentfix">Go to <b>Options</b> under <b>Tools</b></li> <li class="indentfix">Click <b>Queueing</b> tab</li> <li class="indentfix">Configure maximum number of concurrent uploads and downloads</li> <li class="indentfix">Click<b> OK</b></li> </ul> <p>In conclusion, you've met S3 Browser. Get the Pro version to get the full benefits of the tools and options it offers developers.</p> Learn how to use the SS3 Browser and how to benefit from it. https://cdn.ttgtmedia.com/visuals/digdeeper/3.jpg https://www.techtarget.com/searchaws/tip/S3-Browser-Amazons-adaptable-data-storage-interface Wed, 16 Apr 2014 14:07:41 GMT <p>The future of the cloud is likely to be based on adding hosted services, accessible through RESTful, Web-friendly application programming interfaces, to provide special enhancements to solve enterprise and developer problems. Amazon's caching, Database Management <a href="https://www.techtarget.com/searchcloudcomputing/definition/Software-as-a-Service">Software as a Service</a> and virtual desktop services are all examples of this trend, which allows developers to deploy machine images that are more than simple <a href="https://www.techtarget.com/searchcloudcomputing/definition/Infrastructure-as-a-Service-IaaS">Infrastructure as a Service</a> (IaaS) but instead are integrated with special cloud-based features.</p> <p>AppStream and Kinesis are two more such services, and these two are aimed at making the cloud into something totally different -- something that can support enormous data streams and challenging <a href="https://www.techtarget.com/whatis/definition/GUI">graphical user interface</a>&nbsp; presentation applications. Separately, AppStream and Kinesis are significant, but together they could be compelling. Combined with the other <a href="https://www.techtarget.com/searchaws/definition/Amazon-Web-Services">Amazon Web Services</a> (AWS) evolutions, they could be game changing.</p> <p>Data generated online isn't the only form of <a href="https://www.techtarget.com/searchdatamanagement/definition/big-data">big data</a>, but it's a form that has special significance because the sources are distributed and the volume is potentially enormous because of the sheer number of sources. In many applications, developers are forced to create massive aggregation networks to collect data and elastic processing frameworks to process it while adjusting to variations in the data volumes, and to make that whole structure low-delay and high-availability at the same time. That's a tall order, given that any given company may have only one data-flow-intensive application.</p> <p>What Amazon is doing with Kinesis is creating a Web service that can marshal elastic resources to deal with distributed or single-stream high-volume data flows. The <a href="https://twitter.com/" target="_blank" rel="noopener">Twitter</a> example is often given, but it may diminish the utility of Kinesis in the eyes of the corporate user. Nearly any financial trading application; much large-scale transaction processing; and in particular, machine-to-machine and Internet of Things traffic fit this data flow model.</p> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> What Amazon is doing with Kinesis is creating a Web service that can marshal elastic resources to deal with distributed or single-stream high-volume data flows. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> <p>Kinesis lets developers define any number of source points and then associate them with processes hosted in an elastic way in Amazon's cloud. The linkage between source points and host processes is a <i>stream</i>, and a stream can be defined to include any number of source points and associate with any number of processes. Normally the streams would replicate across Amazon's availability zones for high reliability. The result is an architecture for developing data-flow-critical applications that would likely be totally impractical on anything but a public cloud platform. Kinesis thus takes buyers and Amazon out of the move-to-the-cloud model and into a model that builds for the cloud.</p> <p>Obviously Kinesis isn't a light-hearted financial commitment for buyers, but is designed for applications that demand exceptional data handling performance and high elasticity. Compared with private dedicated infrastructure, Kinesis could be a bargain, especially when the data volumes are highly variable. There are even applications using Kinesis as the framework for delivering back-end transactions from a Web store or set of Web applications to transaction processing systems. As Kinesis evolves, it's likely to be available in simpler forms with lower cost and even wider applications. One of the powerful truths about the public cloud is that it can make very efficient use of resources even when individual application and user commitments are small. When Amazon is ready, Kinesis could be a mass market tool.</p> <p>Amazon's second extension, AppStream, appears to be a productization of the content-rendering Amazon uses for its Kindle interfaces. One of the challenges of nearly all gaming applications and many video production and even graphic simulation and display applications is that they rely on a high-performance graphics-processing unit for execution. This kind of technology isn't found on most business laptops, much less on tablets or smartphones. AppStream creates a proxy in the cloud between the application and the device, and this proxy renders the graphics as a <a href="https://www.techtarget.com/searchvirtualdesktop/definition/GPU-graphics-processing-unit">graphics processing unit</a> (GPU) would, and translates the result into a simple display stream that can be handled by nearly any modern computer, tablet or smartphone.</p> <p>The obvious virtue of AppStream is its ability to allow graphics-intensive applications to be run from devices without the special GPU tools. This can be critical to gaming, of course, but this example may again trivialize the concept in the minds of the enterprise, which may be in fact the biggest customer. In healthcare, the financial industry, industrial engineering and design, and other applications that involve graphic rendering of data or graphic depiction of circuits or structures, AppStream can open the world of tablets and smartphones to graphic applications.</p> <p>A second benefit of AppStream is that it's possible to deliver the data to a client browser or app that runs on multiple platforms without changing the underlying application. While Amazon's material isn't making the connection obvious, it seems likely that AppStream could be combined with the Amazon WorkSpaces virtual desktop for corporate use, and even be used in conjunction with both Workspaces and Kinesis to visualize large and complex data flows. The value of this combination in industries like healthcare and financial trading should be obvious.</p> <p>That, perhaps, is the real story with Kinesis and AppStream. Amazon is working hard to build a community of software services deployed in the cloud and available to users and developers to build their own applications. This is making Amazon into more than a cloud; it's creating a distributed software operating system that developers can write for. Over time, this will not only encourage more Amazon partners in their network, it will make Amazon a unique platform for cloud-based applications, not just another IaaS player. That certainly appears to be Amazon's long-term plan.</p> Learn why AppStream and Kinesis are different kinds of data flow services. https://cdn.ttgtmedia.com/visuals/digdeeper/5.jpg https://www.techtarget.com/searchaws/tip/Amazons-data-flow-PaaS-AppStream-and-Kinesis Mon, 27 Jan 2014 10:19:00 GMT <p><b>Amazon recently launched </b><b>Amazon Kinesis</b><b>, a managed service for rapid processing of large amounts of data. Do you have any thoughts about the significance of this service and where it might be most useful?</b></p> <p>Amazon Kinesis is a pipeline for processing data streams in near-real time. It's similar to <a href="http://incubator.apache.org/projects/storm.html" target="_blank" rel="noopener">Apache Storm</a>, but it's hosted and scaled entirely by <a href="https://www.techtarget.com/searchaws/definition/Amazon-Web-Services">Amazon Web Services</a>.</p> <p><a href="https://www.techtarget.com/searchcloudcomputing/tip/Amazon-Kinesis-benefits-and-alternatives">Kinesis offers</a> a complete solution for processing data in near real time (within 10 seconds). Kinesis doesn't really offer any new concepts, but the managed version can serve as a complete suite for specific use cases.</p> <p>Some examples include:</p> <ul type="disc" class="default-list"> <li class="indentfix">Processing log data in real time (to find possible errors and alert IT staff when problems occur)</li> <li class="indentfix">Receiving real-time analysis of application usage</li> <li class="indentfix">Setting up real-time alerts for notification when someone mentions a particular company on Twitter, Facebook or Google+</li> <li class="indentfix">Monitoring real-time news sources for content using specific keywords, then delivering that content to a mobile device</li> </ul> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> The real advantage of Amazon Kinesis is that it makes it easier to build new services from scratch. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> <p>The type of processing that Kinesis does is identical to processing that can be done with other Amazon services. You can already use Amazon's <a href="https://www.techtarget.com/searchaws/feature/Amazon-Web-Services-product-directory">Simple Queue Service</a>, Simple Notification Service and <a href="https://www.techtarget.com/searchcloudcomputing/definition/autoscaling">autoscaling</a> capability to process real-time streams of data (in fact, at my company, Newstex, this is exactly what we do). The real advantage of Amazon Kinesis is that it makes it easier to build new services from scratch, providing a completely managed service for the entire process.</p> <p>Ultimately, Amazon Kinesis will be most useful for addressing big-data problems, such as:</p> <ul type="disc" class="default-list"> <li class="indentfix">Processing log data</li> <li class="indentfix">Processing social media streams for specific terms or key phrases</li> <li class="indentfix">Identifying trends in stock prices</li> <li class="indentfix">Analyzing real-time sales statistics</li> </ul> <p>Even if you don't have a large amount of data to process, this type of architecture helps with fault tolerance, as well as scaling. So, there are many reasons to use Amazon Kinesis or something similar if you're worried about real-time processing of any stream of data.</p> In this expert answer, Chris Moyer offers advice on using Amazon Kinesis for near-real-time processing of streaming big data. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/answer/Amazon-Kinesis-When-to-use-Amazons-new-big-data-processing-service Mon, 06 Jan 2014 13:59:00 GMT <p>One of the leading announcements at the 2013 Amazon Web Services (AWS) re:Invent conference was the launch of Amazon AppStream, a service that lets developers build sophisticated applications that users can stream from the cloud. Initially available for limited preview, this technology promises to override device limitations such as processing power and memory storage.</p> <p>In announcing AppStream, Andy Jassy, Amazon senior vice president of AWS, said AppStream applications would seem to come directly from end-user devices. The quality of cloud streaming, he said, was that good.</p> <p>Following are a few answers to frequently asked questions about <a href="http://aws.amazon.com/appstream/" target="_blank" rel="noopener">Amazon AppStream</a> to help IT teams decide whether the new service meets their needs.</p> <section class="section main-article-chapter" data-menu-title="How does Amazon AppStream work?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>How does Amazon AppStream work?</b></h2> <blockquote class="pullquote alignRight"> <div> <div> <div> <span class="pullquoteContent">Amazon AppStream's primary benefit is its ability to stream resource-intensive apps to mass-market devices.</span> </div> </div> </div> </blockquote> <p>AppStream allows AWS developers to stream high-powered, resource-demanding applications from the cloud, giving end users access to complex apps such as three-dimensional games and high-definition content on a wide variety of devices.</p> <p>Developers have the option to stream whole applications or just the fragments that demand more computing power. That capability makes it possible to write all or portions of the code, and to make in-the-moment changes to the software.</p> </section> <section class="section main-article-chapter" data-menu-title="What is the Amazon AppStream STX Protocol?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>What is the Amazon AppStream STX Protocol?</b></h2> <p>It is a proprietary protocol used to stream application audio and video over a variety of network conditions. It also sends user input over <a href="https://www.techtarget.com/searchnetworking/definition/TCP">TCP</a> back to the app. Network conditions are measured throughout and the data is sent back to the server so the streaming video and audio can be altered accordingly.</p> </section> <section class="section main-article-chapter" data-menu-title="What applications are best suited to Amazon AppStream?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>What applications are best suited to Amazon AppStream?</b></h2> <p>The service is designed for applications that demand a lot of computing power or storage space. For that reason, it's best suited for apps that are too heavy to run natively. That category includes games, interactive videos, photo-editing programs and 3D-rendering applications. It's also possible to host a portion of these apps natively and to stream the rest from the cloud. That said, applications that don't respond well to latency wouldn't be well-suited to streaming, which always involves wait time to load.</p> </section> <section class="section main-article-chapter" data-menu-title="What are the benefits of Amazon AppStream?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>What are the benefits of Amazon AppStream?</b></h2> <p>The primary benefit is the service's ability to stream resource-intensive apps to mass-market devices. This capability resolves developers' ongoing struggle to build apps for a wide variety of hardware and all its inherent limitations. Ultimately, this saves the time and expense associated with coding for different devices.</p> <p>AppStream also simplifies updates, which no longer require any user initiation. Developers have only to provide a new version of the app to AppStream and the software will automatically update on the device. Also, because the data streams directly from the cloud, users will be spared time-consuming downloads and installations. In terms of security, AppStream apps are stored in AWS data centers and, for that reason, aren't subject to the same vulnerabilities as native apps (for example, reverse engineering and theft).</p> </section> <section class="section main-article-chapter" data-menu-title="Are there any possible drawbacks to Amazon AppStream?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>Are there any possible drawbacks to Amazon AppStream?</b></h2> <p>Latency is always a factor where streaming is concerned and it will take the AppStream application some time to react to user commands. Amazon claims to have optimized the capture and encoding process in order to minimize latency as much as possible, but variable network conditions will sometimes cause a lag in response time when apps run over the Internet.&nbsp; That also means AppStream apps won't be able to run offline. &nbsp;A workaround would be to run part of the application natively.</p> </section> <section class="section main-article-chapter" data-menu-title="How does Amazon AppStream pricing work?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>How does Amazon AppStream pricing work?</b></h2> <p>During the limited preview, the service is free and you can run up to 10 simultaneous streams. When AppStream is commercially available, Amazon will collect a fee for every hour a customer streams content.</p> </section> <section class="section main-article-chapter" data-menu-title="How can companies get started with Amazon AppStream?"> <h2 class="section-title"><i class="icon" data-icon="1"></i><b>How can companies get started with Amazon AppStream?</b></h2> <p>At this writing, <a href="http://aws.amazon.com/appstream/limited-preview/" target="_blank" rel="noopener">developers can request access</a> for the limited preview of AppStream. Amazon said that, as space becomes available, it will fulfill requests with instructions on accessing the limited preview.</p> </section> In this feature, contributor Caroline de Lacvivier answers frequently asked questions about Amazon Web Services' new developer tool, Amazon AppStream. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/feature/Learn-about-using-Amazon-AppStream-to-stream-resource-gobbling-apps Mon, 06 Jan 2014 10:44:00 GMT <p>At the 2013 Amazon Web Services re: Invent conference, Amazon announced the launch of Amazon Kinesis, its service for real-time big data processing. Just recently, the technology has become broadly available for use.</p> <p>Terry Hanold, <a href="https://www.techtarget.com/searchaws/definition/Amazon-Web-Services">AWS</a> vice president of new business initiatives, called the product a response to a shortcoming in <a href="https://www.techtarget.com/searchdatamanagement/definition/Hadoop">Hadoop</a> and other technologies that process data in batches. While those technologies are capable of handling large quantities of data, Amazon says, they don't enable real-time, ongoing decision making on <a href="https://www.techtarget.com/searchnetworking/definition/data-streaming">data that's constantly streaming</a>. Amazon Kinesis will provide such service on AWS' standard pay-as-you-go basis.</p> <p>Following are answers to frequently asked questions about <a href="https://www.techtarget.com/searchcloudcomputing/tip/Amazon-Kinesis-benefits-and-alternatives">Amazon Kinesis</a> to help you decide if it's a fit for your goals.</p> <p><b>What is Amazon Kinesis?</b></p> <p>Amazon Kinesis is a managed service for real-time processing of streaming data on a large scale. It helps build applications that respond quickly to changes in a data stream.</p> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> Kinesis applications allow users to make in-the-moment operational decisions. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> <p><b>How can Amazon Kinesis be used?</b></p> <p>Kinesis is ultimately meant to simplify the process of building apps that operate on real-time data. According to Amazon, it's capable of collecting and processing hundreds of <a href="https://www.techtarget.com/searchstorage/definition/terabyte">terabytes</a> per hour from hundreds of thousands of sources. In other words, it helps manage information that streams in constantly and at high volume. That category can include anything from operating logs to financial transactions, from social media feeds to clickstreams.</p> <p>Kinesis applications allow users to make in-the-moment operational decisions, such as building real-time dashboards; generating alerts; and sending data to other big data services, such as Amazon Redshift or <a href="https://www.techtarget.com/searchaws/definition/Amazon-Dynamo-Database-DDB">Amazon Dynamo Database</a> (DynamoDB). Because the streamed data is stored in several AWS availability zones for 24 hours, users have a window of time in which to read and analyze the data, as well as move it into long-term storage.</p> <p><b>Who should be using Amazon Kinesis?</b></p> <p>Generally speaking, the service is intended for developers building real-time applications, particularly those dealing with large volumes of data. James Staten, a Forrester Research analyst, has said that Kinesis would be particularly useful for companies such as Autodesk Inc., which is looking to bring computer-aided design (CAD) apps and three-dimensional rendering tools to the cloud. Amazon also specified the types of data that might be in an Amazon Kinesis use case: IT infrastructure log data, social media, game development, Web clickstream data, application logos and market data feeds.</p> <p><b>What benefits does Amazon Kinesis offer?</b></p> <p>Amazon Kinesis integrates with Redshift, Dynamo DB and <a href="https://www.techtarget.com/searchaws/definition/Amazon-Simple-Storage-Service-Amazon-S3">Amazon Simple Storage Service</a>, or (S3, as well as third-party products, a perk for developers who don't want to abandon their personal data-processing methods. As to security, Kinesis authenticates users through cryptographic methods that prevent unauthorized parties from accessing data.</p> <p><b>What's the pricing model for Amazon Kinesis?</b></p> <p>Like other AWS services, Kinesis is priced on a pay-as-you-go basis. Users are billed based on the amount of data processed and the way in which it's packaged. For further information, the <a href="http://aws.amazon.com/kinesis/pricing/" target="_blank" rel="noopener">Amazon Kinesis webpage</a> goes more in-depth.</p> <p><b>How do I get started?</b></p> <p>You can generate an Amazon Kinesis stream in one of two ways: You can go through the AWS Management Console or you can make an API request. Unsure how? Amazon will <a href="http://aws.amazon.com/kinesis" target="_blank" rel="noopener">walk you through it</a>.</p> Amazon Kinesis was launched at the 2013 AWS conference. Here are some answers to frequently asked questions to help you decide if it's right for you. https://cdn.ttgtmedia.com/visuals/digdeeper/4.jpg https://www.techtarget.com/searchaws/feature/FAQ-AWS-takes-on-real-time-processing-with-Amazon-Kinesis Fri, 20 Dec 2013 10:27:00 GMT 60 webmaster@techtarget.com