USER
implement the code for this paper which accepts image and display lane after training the specific model "Abstract
An increasing safety and reducing road
accidents, thereby saving lives are one of great interest
in the context of Advanced Driver Assistance Systems.
Apparently, among the complex and challenging tasks
of future road vehicles is road lane detection or road
boundaries detection. It is based on lane detection
(which includes the localization of the road, the
determination of the relative position between vehicle
and road, and the analysis of the vehicle’s heading
direction). One of the principal approaches to detect
road boundaries and lanes using vision system on the
vehicle. However, lane detection is a difficult problem
because of the varying road conditions that one can
encounter while driving. In this paper, a vision-based
lane detection approach capable of reaching real time
operation with robustness to lighting change and
shadows is presented. The system acquires the front
view using a camera mounted on the vehicle then
applying few processes in order to detect the lanes.
Using a pair of hyperbolas which are fitting to the
edges of the lane, those lanes are extracted using
Hough transform. The proposed lane detection system
can be applied on both painted and unpainted road as
well as curved and straight road in different weather
conditions. This approach was tested and the
experimental results show that the proposed scheme
was robust and fast enough for real time requirements.
Eventually, a critical overview of the methods were
discussed, their potential for future deployment were
assist.
Keywords
Driver Assistance System, Lane detection, computer
vision, intelligent vehicles
I. INTRODUCTION
In intelligent transportation systems, intelligent
vehicle cooperate with smart infrastructure to achieve a
safer environment and batter traffic conditions.
Although, a more convincing reason to build
intelligent vehicles is to improve the safety conditions
by the entire or partial automation of driving tasks.
Among these tasks, the road detection took an
important role in driving assistance systems that
provides information such as lane structure and vehicle
position relative to the lane. However, the most
compelling reason for adding autonomous capability to
vehicles that to ensure the safety requirement. Vehicle
crashes remain the leading cause of accident death and
injuries in Malaysia and Asian countries claiming tens
of thousands of lives and injuring millions of people
each year. Most of these transportation deaths and
injuries occur on the nation’s highways. The United
Nations has ranked Malaysia 30th among countries
with the highest number of fatal road accidents,
registering an average of 4.5 deaths per 10,000
registered vehicles [1]. Therefore, a system that
provides a means of warning the driver to the danger
has the potential to save a considerable number of
lives. One of the main technology involves in these
takes computer vision which become a powerful tool
for sensing the environment and has been widely used
in many application by the intelligent transportation
systems (ITS) In many proposed systems[2], the lane
detection consists of the localization of specific
primitives such as the road markings of the surface of
painted roads. This restriction simplifies the process of
detection, nevertheless, two situations can disturb the
process: the presence of other vehicles on the same
lane occluded partially the road markings ahead of the
vehicle are the presence of shadows caused by trees,
buildings etc. This paper presents vision- based
approach capable of reaching a real time performance
in detection and tracking of structured road boundaries
(painted or unpainted lane markings) with slight
curvature, which is robust enough in presence of
shadow conditions. Road boundaries are detected by
fitting a parallel hyperbola pairs to the edges of the
82
lane after applying the edge detection and Hough
transform. The vehicle is supposed to move on a flat
and straight road or with slow curvature.
II. RELATED WORK
Safety is the main objective of all the road lane
detection systems due to the reason is that most of the
vehicle road accident happens because of the driver
miss leading of the vehicle path. Therefore, currently
many different vision-based road detection algorithms
have been developed to avoid vehicle crash on the
road. Among these algorithms the GOLD system
developed by Broggi, it uses an edge-based lane
boundary detection algorithm [3]. The acquired image
is remapped in a new image representing a bird’s eye
view of the road where the lane markings are nearly
vertical bright lines on a darker background. Specific
adaptive filtering is used to extract quasi vertical bright
lines that concatenated into specific larger segments.
Kreucher C. propose in [4] the LOIS algorithm as a
deformable template approach. A parametric family of
shapes describes the set of all possible ways that the
lane edges could appear in the image. A function is
defined whose value is proportional to how well a
particular set of lane shape parameters matches the
pixel data in a specified image. Lane detection is
performed by finding the lane shape that maximizes
the function for the current image. The Carnegie
Mellon University proposes the RALPH system, used
to control the lateral position of an autonomous vehicle
[5]. It uses a matching technique that adaptively adjusts
and aligns a template to the averaged scan line
intensity profile in order to determine the lane’s
curvature and lateral offsets. The same university
developed another system called AURORA which
tracks the lane markers present on structured road
using a color camera mounted on the side of a car
pointed downwards toward the road [6]. A single scan
line is applied in each image to detect the lane markers.
An algorithm destined to painted or unpainted road is
described in [7]. Some color cues were used to conduct
image segmentation and remove the shadow.
Assuming that the lanes are normally long with smooth
curves then theirs boundaries can be detected using
Hough transformation applied to the edge image. A
temporal correlation assumed between successive
images is used in the following phase. Three-feature
based automatic lane detection algorithm (TFALDA)
[8] is primarily intended for automatic extraction of the
lane boundaries without manual initialization or a
priori information under different road environments
and real-time processing. It is based upon similarity
match in a three dimensional (3-D) space spanned by
the three features of a lane boundary starting position
direction (or orientation), and its gray-level intensity
features comprising a lane vector are obtained via
simple image processing. LANA algorithm [9] was
based on novel set of frequency domain features that
captures relevant information concerning the
magnitude and orientation of spatial edges extracted by
8*8(DCT). The hyperbola-pair model is deformable
template, and is developed on the base of Kluge’s work
[10]. The origin model is suitable for single-side lane
markings firstly, but in the work by Wang and Chen
[11]. It was emphasized that some parameters of the
model was the same, if the lanes were parallel on the
same road. The considerable feature is also used in our
algorithms, to form an extended equation to fit the road
shape.
III. ENVIRONMANTAL VARIABILTY
In addition to the intended application of the vision
lane detection system, it is important to evaluate the
type of conditions that are expected to be countered.
Road markings can vary greatly not only between
regions, but also over nearby stretches of road. Roads
can be marked by well-defined solid lines, segmented
lines, circular reflectors, physical barriers, or even
nothing at all. The road surface can be comprised of
light or dark pavements or combinations. An example
of the variety of road conditions can be seen in Fig. 1,
some roads shows a relatively simple scene with both
solid line and dashed line lane markings. Lane position
in this scene can be considered relatively easy because
of the clearly defined markings and uniform road
texture. But in other complex scene in which the road
surface varies and markings consist of circular
reflectors as well as solid lines the lane detection will
not be an easy task. Furthermore, shadowing obscuring
road markings makes the edge detection phase more
Figure1. Different road scene
83
complex. Along with the various types of markings
and shadowing, weather conditions, and time of day
can have a great impact on the visibility of the road
surface as shown in Figure 1. All these circumstances
must be efficiently handled in order to achieve an
accurate vision system.
IV. OVER VIEW OF OUR ALGORITHM
Figure 2. Algorithm over view
A CCD camera is fixed on the front-view mirror to
capture the road scene. To simplify the problem, we
assume that it is setup to make the baseline horizontal,
which assures the horizon in the image, is parallel to
the X-axis. Otherwise, we can adjust the image using
the calibration data of the camera to make it. Each lane
boundary marking, usually a rectangle (or
approximate) forms a pair of edge lines. In this paper,
it was assumed that the input to the algorithm was a
620x480 RGB color image. Therefore the first thing
the algorithm does is to convert the image to a
grayscale image in order to minimize the processing
time. Secondly, as presence of noise in the image will
hinder the correct edge detection. Therefore, we apply
(F.H.D.) algorithm [13] to make the edge detection
more accurate. Then the edge detector is used to
produce an edge image by using canny filter with
automatic thresholding to obtain the edges, it will
reduce the amount of learning data required by
simplifying the image edges considerably. Then edged
image sent to the line detector after detecting the edges
which will produces a right and left lane boundary
segment. The projected intersection of these two line
segments is determined and is referred to as the
horizon. The lane boundary scan uses the information
in the edge image detected by the Hough transform to
perform the scan. The scan returns a series of points on
the right and left side. Finally pair of hyperbolas is
fitted to these data points to represent the lane
boundaries. For visualization purposes the hyperbolas
are displayed on the original color image. The
algorithm structure is shown in Figure 2.
A. Image Capturing
The input data is a color image sequence taken
from a moving vehicle. A color camera is mounted
inside the vehicle at the front-view mirror along the
central line. It takes the images of the environment in
front of the vehicle, including the road, vehicles on the
road, roadside, and sometimes incident objects on the
road. The on-board computer with image capturing
card will capture the images in real time (up to 30
frames/second), and save them in the computer
memory. The lane detection system reads the image
sequence from the memory and starts processing. A
typical scene of the road ahead is depicted by Figure 1.
In order to obtain good estimates of lanes and improve
the speed of the algorithm, the original image size was
reduced to 620x480 pixels by Gaussian pyramid.
B. Conversion to Gray Scale
To retain the color information and segment the
road from the lane boundaries using the color
information this proved difficulties on edge detection
and also it will effect the processing time. In practice
the road surface can be made up of many different
colors due to shadows, different pavement style or age,
which causes the color of the road surface and lane
markings to change from one image region to another.
Therefore, color image are converted into grayscale.
However, the processing of grayscale images becomes
84
minimal as compared to a color image. This function
transforms a 24-bit, three-channel, color image to an 8-
bit, single-channel grayscale image by forming a
weighted sum of the Red component of the pixel value
* 0.3 +Green component of the pixel value * 0.59 +
Blue component for the pixel value *0.11 the output
is the gray scale value for the corresponding pixel[12].
C. Noise Reduction
Noise is a real world problem for all systems and
computer vision is no exception. The algorithms
developed must either be noise tolerant or the noise
must be eliminated. As presence of noise in our system
will hinder the correct edge detection, so that noise
removal is a pre requisite for efficient edge detection
with the help of (F.H.D.) algorithm [13] that removes
strong shadows from a single image. The basic idea is
that a shadow has a distinguished boundary. Removing
the shadow boundary from the image derivatives and
reconstructing the image should remove the. A shadow
edge image can be created by applying edge-detection
on the invariant image and the original image, and
selecting the edges that exist in the original image but
not in the invariant image and to reconstruct the
shadow free image by removing the edges from the
original image using a pseudo-inverse filter.
D. Edge Detection
Lane boundaries are defined by sharp contrast
between the road surface and painted lines or some
type of non-pavement surface. These sharp contrasts
are edges in the image. Therefore edge detectors are
very important in determining the location of lane
boundaries. It also reduces the amount of learning data
required by simplifying the image considerably, if the
outline of a road can be extracted from the image. The
edge detector implemented for this algorithm and the
one that produced the best edge images from all the
edge detectors evaluated was the ‘canny’ edge detector
[14]. To find the maxima of the partial derivative of the
image function I in the direction orthogonal to the edge
direction, and to smooth the signal along the edge
direction. Thus Canny's operator looks for the maxima
of :
2 2
2 2
1
exp
2 2
x y Gσ
πσ σ
⎡ ⎤ + = −⎢ ⎥ ⎣ ⎦
(1.1)
Where
*
*
G I
n
G I
∇ = ∇ (1.2)
Figure 3. Canny detection
It was important to have the edge detection
algorithm that could be able to select thresholds
automatically however, the automatic threshold that is
used in the default Canny produced far too much edge
information. Making a slight modification to the edge
detected produced by canny has given more desirable
results. The only changes necessary were to set the
amount of non-edge pixels to the best value that gives
more accurate edges in different conditions of image
capturing environment. The canny edge detector also
has a very desirable characteristic in that it does not
produce noise like the other approaches
E. Line Detection
The line detector used is a standard Hough
transform [8] with a restricted search space. The
standard Hough transforms searches for lines using the
equation shown in Figure 4:
Figure 4 Hough Transform
This of course looks for all possible lines in the image,
Figure 4. Hough Transform
When in reality we can reject any line that falls
outside a certain region [15]. For example a horizontal
line is probably not the lane boundary and can be
rejected. The restricted Hough transform was modified
to limit the search space to 45° for each side. Also the
input image is split in half yielding a right and left side
of the image. Each the right and left sides are searched
separately returning the most dominant line in the half
image that falls with in the 45° window. The horizon is
simply calculated using the left and right Hough lines
and projecting them to their intersection. The
85
horizontal line at this intersection is referred to as the
horizon.
F. Lane Boundary Scan
The lane boundary scan phase uses the edge
image the Hough lines and the horizon line as input.
The edge image is what is scanned and the edges are
the data points it collects. The scan begins where the
projected Hough lines intersect the image border at the
bottom of the image. Once that intersection is found, it
is considered the starting point for the left or right
search, depending upon which intersection is at hand.
From the starting point, the search begins a certain
number of pixels towards the center of the lane and
then proceeds to look for the first edge pixel until
reaching a specified number of pixels after the
maximum range. The range will be set to the location
of the previously located edge pixel plus a buffer
number of pixels further
Figure 5. Image coordinates
For the starting condition, the search will already be
at the left- or right-most border, as shown in Figure 5
.so the maximum range will be the border itself. The
extra buffer zone of the search will help facilitate the
following of outward curves of the lane boundaries.
The lane points are organized into two lists expressed
as L(l) and L(r)
() () () () () () ()
11 22 {( , ),( , ),...........,( , )} l ll ll ll L uv uv uv = m m (1.3)
() () () () () () ()
11 22 {( , ),( , ),...........,( , )} r rr rr rr L uv uv uv = m m (1.4)
Figure 6 shows the lane boundaries detection.
G. Hyperbola Fitting
The hyperbola pair fitting phase uses the two
vectors of data points from the lane scan as input. A
least squares technique is used to fit a hyperbola to the
data. One hyperbola is fit to each of the vectors of data
points; however, they are solved in simultaneously due
to the fact that they are a pair model. The parameters of
the two hyperbolas are related because they must
converge to the same point, due to the geometry of the
roadway as shown in Figure7. The formula for
expressing the lane boundary as a hyperbola [11],
given the road boundary point (u, v) in image plane:
( ) k u bv h c
u h = + −+ − (1.5)
u and v are the x- and y-coordinate in the image
reference frame, h is the Y-coordinate of the horizon in
the image reference frame, and k, b and c are the
parameters of the curve, which can be calculated from
the shape of the lane.
Figure 6. Lane boundary detection
Figure 7. Hyperbola fitting
V. EXPERIMENTAL RESULTS
In this paper, the algorithm was implemented in
a HP Intel ® Core TM (2) 2.0 GHz computer using
Matlab 7.1. A database including a growing number of
Left Lane
Right Lane
X
Y
86
image and video frames is set up for the experiment
.All these images are taken in highways and normal
roads, dashed markings, straight and curved roads in
different environmental conditions (sunny, cloudy,
nighttime, shadowing, rainy). Figure 8 illustrates the
performance of the system under different road
conditions. Additionally, in Figure 9 we can observe
that the lane boundaries are successfully extracted,
which indicate the robustness and real-time
performance of the algorithm. However, still some
problems did not solved yet such as sharp curves in the
foreground of the image and the accurate detection of
the lanes under heavy rain also the captured frames are
not that stabile due to the vehicle movement and
therefore, we need to improve the algorithm to
overcome these problems.
Figure 8. Average accuracy rates of lane detection in different
road conditions.
VI. SUMMARY AND CONLUSION
In this paper, a real time lane detection algorithm
based on video sequences taken from a vehicle driving
on highway was proposed. As mentioned above the
system uses a series of images. Out of these series
some of the different frames used are shown in the lane
detection algorithm, (F.H.D.) algorithm conduct image
segmentation and remove the shadow of the road.
Since the lanes are normally long and smooth curves,
we consider them as straight lines within a reasonable
range for vehicle safety. The lanes were detected using
Hough transformation with restricted search area. The
proposed lane detection algorithm can be applied in
both painted and unpainted road, as well as slightly
curved and straight road as shown in Figure 9. There
remained some problems in the lane detection due to
shadowing and the Hough line and horizon overlaid,
then in the lower left edge with the lane boundary
points overlaid and in few cases lane scan fails to track
the correct lane.
Figure 9. Road lane detection in different scene
ACKNOWLEDGMENT
The authors of this paper would like to thank the
research management center at IIUM for their financial
support under Research Endowment Grant (Type B)"ASSISTANT
To implement the proposed lane detection algorithm, we need to follow several steps to preprocess the image, detect edges, apply the Hough transform to find lines, and then fit hyperbolas to the detected lane boundaries. Below is a Python code snippet using OpenCV and NumPy which follows the described steps:
```python
import cv2
import numpy as np
# Hyperparameters
RESIZE_WIDTH = 620
RESIZE_HEIGHT = 480
CANNY_LOW_THRESHOLD = 50
CANNY_HIGH_THRESHOLD = 150
HOUGH_RHO = 1
HOUGH_THETA = np.pi / 180
HOUGH_THRESHOLD = 15
HOUGH_MIN_LINE_LENGTH = 40
HOUGH_MAX_LINE_GAP = 20
def to_grayscale(image):
return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
def remove_noise(image):
return cv2.GaussianBlur(image, (5, 5), 0)
def detect_edges(image):
return cv2.Canny(image, CANNY_LOW_THRESHOLD, CANNY_HIGH_THRESHOLD)
def hough_transform(edges):
return cv2.HoughLinesP(edges, HOUGH_RHO, HOUGH_THETA, HOUGH_THRESHOLD, np.array([]),
minLineLength=HOUGH_MIN_LINE_LENGTH, maxLineGap=HOUGH_MAX_LINE_GAP)
def draw_lines(image, lines):
line_image = np.zeros_like(image)
if lines is not None:
for line in lines:
for x1, y1, x2, y2 in line:
cv2.line(line_image, (x1, y1), (x2, y2), (255, 0, 0), 10)
return line_image
def fit_hyperbola(points):
# For simplicity, use polynomial fitting as a placeholder for hyperbola fitting
points = np.array(points)
if len(points) > 0:
poly = np.polyfit(points[:, 1], points[:, 0], 2)
return poly
return None
def draw_hyperbolas(image, left_fit, right_fit):
if left_fit is not None or right_fit is not None:
y_vals = np.linspace(0, image.shape[0] - 1, image.shape[0])
left_points = [(int(np.poly1d(left_fit)(y)), int(y)) for y in y_vals]
right_points = [(int(np.poly1d(right_fit)(y)), int(y)) for y in y_vals]
for point in left_points:
cv2.circle(image, point, 5, (0, 255, 0), -1)
for point in right_points:
cv2.circle(image, point, 5, (0, 255, 0), -1)
return image
def preprocess(image):
resized_image = cv2.resize(image, (RESIZE_WIDTH, RESIZE_HEIGHT))
gray_image = to_grayscale(resized_image)
noise_free_image = remove_noise(gray_image)
edges = detect_edges(noise_free_image)
return edges
def lane_detection_pipeline(image):
edges = preprocess(image)
lines = hough_transform(edges)
line_image = draw_lines(image, lines)
# Placeholder for actual point extraction for hyperbola fitting
left_points = [pt for line in lines for pt in line if line is not None and pt[0][0] < image.shape[1] // 2]
right_points = [pt for line in lines for pt in line if line is not None and pt[0][0] > image.shape[1] // 2]
left_fit = fit_hyperbola(left_points)
right_fit = fit_hyperbola(right_points)
final_image = draw_hyperbolas(image.copy(), left_fit, right_fit)
return final_image
def main():
cap = cv2.VideoCapture('path_to_video.mp4')
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
processed_frame = lane_detection_pipeline(frame)
cv2.imshow('Lane Detection', processed_frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
if __name__ == "__main__":
main()
```
### Explanation:
1. **Image Preprocessing:**
- Convert the image to grayscale to reduce computational load and focus on intensity changes.
- Remove noise using Gaussian blur.
2. **Edge Detection:**
- Use the Canny edge detection algorithm to highlight the edges in the grayscale image.
3. **Hough Transform:**
- Apply a Hough transform to detect lines within the edges.
4. **Line Drawing:**
- Draw the detected lines onto a separate image to visualize.
5. **Hyperbola Fitting (Placeholder):**
- Extract points which theoretically belong to the left and right lanes.
- Use polynomial fitting as a placeholder for hyperbola fitting. In practice, you might need to adjust this to actually fit hyperbolas.
6. **Visualization:**
- Draw the fitted curves (hyperbolas) on the original image.
7. **Video Input Handling:**
- Read from a video file or real-time camera feed.
- Process each frame using the lane detection pipeline and show the result.
### Note:
This code provides a simplified version fitting polynomials instead of hyperbolas due to complexity. For real deployment, further refinement such as robust hyperbola fitting and advanced noise handling might be needed.