VisiMark1_0: An Assistance Tool for Evaluating Robustness of Video Watermarking Algorithms

Published Online April 2013 in MECS

The technology of embedding and retrieving information into and from video data is video watermarking. Literature survey proposes a concoction of robust and fragile watermarking methods for resolving proof ownership problems [1, 2], copyright protection [3, 4] and video authentication [5]. Various algorithms have been proposed in the scientific literature by numerous authors for robust watermark embedding in video.

The crucial components implicated in robust watermarking are watermark embedding, attack, and watermark extraction or detection. In the first phase of watermark embedding, a secure watermark sign (Text, Image or Audio etc) is designed using several technologies like encryption scrambling etc. This secured watermark is then ingrained into an original signal (Video in context with this paper) exploring any of the domains (spatial/frequency/feature etc) of watermarking. Successful embedding algorithms generate the watermarked Video. The third phase is the extraction or detection of the watermark. A triumphant extraction algorithm is one in which the watermark sign could be extracted even after the subjection of an assortment of attacks to the watermarked video. During watermark detection, the watermark detector is specified with a test signal that may be watermarked, attacked or not. The watermark detector reports whether the watermark is present or not on investigating the signal at its input. In this paper we focus on the intermediate ingredient allied to attacks. The proposed evaluation tool contributes towards 9 video watermarking attacks and 3 offerings in the image category. Also the scene change detection algorithm, a user friendly GUI, comparison of two general videos for parameters etc are certain contributions in the entire tool developed. This paper is organized in five sections. The following section elaborates on the literature survey and the techniques offered by various tools like optimark, and checkmark. The third section is a vast section focusing on the features of VisiMark1_0 along with all its implementation particulars. The evaluated results are presented in the subsequent section followed by future scope of the proposed tool and conclusions.

• II.    Literature Survey

  Robustness has been proved to be decisive factor for evaluating the watermarking algorithms. The robustness of a video watermarking algorithm is verified by the diverse attacks it can achieve over and still recover the embedded data. The emergent quantity of attacks against watermarking systems [6], [7] has revealed that additional exploration is imperative to perk up the excellence of existing watermarking algorithms. In 1997 Peticols introduced Stirmark [8,9] that has been the pioneer in benchmarking images involving signal processing attacks like Gaussian filtering and noising , geometrical attacks like flipping, Cropping, Rotation, Random bending , Rotation-Scale, FMLR, sharpening, Random bending, Line removal, JPEG compression , linear transformations, Aspect ratio etc. Optimark [10] was released in 2001. The attacks it handled for .ppm image formats were no attack, line and column removal, scaling, shearing, horizontal flip, rotation, auto crop, auto scale, sharpening, Gaussian, median and jpeg. The further tool that emerged was checkmark [11] in 2001. The features added up in checkmark were the support for color, Projective transformations, Denoising, Wavelet compression , warping, Copy attack ,Template removal attack , Non-linear line removal ,Collage attack, down/up sampling, dithering etc.

• III.    Features of Visimark1_0

• 3.1    Visimark1_0 attacks

• 2.    Frame dropping:-

• There is a diminutive alteration between frames in a shot. Frame dropping is a standard attack adopted by researchers to verify the robustness of the watermarked video. In the VisiMark1_0 project, frame dropping is done with four options which could further be enhanced. Depending on the frame dropping factor (Fr), the frames from the video are dropped. Example when Fr=2 and X1=0

Since unscrambling the video in frames, testing every individual frame with a geometrical/signal processing attack, reconstruction of the video for that attack and then following the same procedure for every attack is a wearisome task for robustness evaluation of a video watermarking algorithm. This section imposes a strong focus on the features of Visimark1_0 that accentuate on the attacks handled in Visimark1_0 and its implementation details. Visimark1_0 reprograms some of the attacks of checkmark as per the requirement. This tool is proposed as assistance for evaluating robustness of video watermarking algorithms.

This section intricate the attacks implemented by Visimark1_0 for avi videos. Particular video category attacks like frame averaging, temporal synchronization etc are applied to videos whereas geometrical attacks like rotation , cropping etc and signal processing attacks like filtering, noising, denoising etc that are applied individually to the frames of the video are elaborated in the section.

VisiMark1_0 Video Category Attacks:-1.    Frame averaging:-

Frame averaging is a commonly used mathematical process for evaluating the robustness of a watermarking algorithm. This significant video watermarking attack has the competence that will disturb the composition of the video. In the VisiMark1_0 experimentation, the average of the current frame and the nearest two neighbors are considered to replace the current frame. Frame Averaging is defined as

Fm(i,j) = [Fm-1(i,j) + Fm(I,j)+ Fm+1(i,j)]/3

where m=1,2…n-2

For i=2 to m

Xi =Xi-1+Fr+1 …. All Fx will be dropped.

• 3.    Frame Swapping:-

• Since frame swapping also agitate the dynamic composition of the video, it is also considered as a robust video watermarking attack. In our experimentation we swapped the neighboring frames of the video to retrieve the watermarked video.

• 4.    Changing the sequence of the scenes in the video:-

• Certain video watermarking algorithms are scene based. So changing the sequence of the scenes in the

• 5.    Changing frame rate of the video or temporal synchronization attack:-

• A preliminary factor in videos is the motion. The measure of the motion of the video displayed is the FPS (Frames per second). Various video watermarking algorithms are based on temporal synchronization hence this is an important attack to test the robustness of the algorithm. Visimark1_0 performs this attack by modifying the FPS in video parameters in steps of 5.

• 6.    Chroma sampling attack:-

Fm(i,j) = Fm-1(i,j) where m=1,2…n-1.

video may disturb the embedded watermarks. The scene change detection algorithm implemented in VisiMark1_0 is summarized in Fig 2. The edges in the frames are detected and their intensity values are compared with a threshold to determine the scene change.

Watermarking algorithms available in the literature are also based on the luminance component since the HVS is less sensitive to variations in the color than in the luminance. So chroma sampling is an important attack that must be evaluated to verify the performance the performance of a watermarking algorithm. Following procedure was adopted for chroma sampling by visimark1 0.

• a)    Convert the rgb frame to Ус dr г

• b)    Down sample the cb component by a factor

• c)    Up sample it by the same factor

• d)    Down sample the cr component by a factor

• e)    Up sample it by the same factor

• f)    Convert back to rgb and convert to frames of video.

• 7.    Fade and dissolve attack:-

• Detection of transitions occurring between the scenes or shots is a regular process in video analysis. Certain watermarking algorithms are based on these transitions and depend on the intensity of the pixels. It is therefore important to apply the fade and dissolve attack to evaluate its performance for robustness. Fade is a process in which the frame of the video sequence gets gradually darkens and is replaced by another frame [12]. When the picture information gradually disappears leaving a blank screen it is a fade out transition. A dissolve is a combination of a fade out and fade in, superimposed on the same film strip [12]. Visimark1_0 achieves the fade and dissolve attack by modifying the pixel intensities of the frames in transition by applying the following logic

• 8.    Motion blurring attack:-

• Motion blurring is the artifact by which fast-moving objects emerge blurred. A filter that blurs the image along a specified axis to give the effect of motion needs to be used to accomplish the motion blurring effect. VisiMark1_0 creates a motion filter with horizontal and vertical motions of 45 and 20, filtering the frames of the video with the motion filter to create a motion blurred attack.

• 9.    Inter frame filtering:-

• Interframe is a frame that is expressed in terms of the neighboring frames. Two Interframe filtering techniques are designed by visimark1_0 is

fading = T£=o(F(.i,f) + vaZ + G * 2))

• a)    Generate frames of the video

Add noise to each frame of the video.

Apply averaging filter to each noisy image

Reconstruct the video

• b)    Generate frames of the video

Apply ideal low pass filter to all the frames of the video.

Compute average of three consecutive frames

VisiMark1_0 Geometric Attacks

The standard transformations performed on images like Affine, Rotate, Scaling, Shearing, Translation and warping are considered as geometrical attacks on the image as they disturb the original composition of the image pixels that may hamper the embedded watermark. Also up sampling and down sampling are considered as standard attacks as they disturb the count of the pixels in the image. VisiMark1_0 exploits commonly used nearest neighbor and Bicubic interpolation techniques. Following geometrical attacks are incorporated in VisiMark1_0.

• 1.    Cropping:-

• Aspect ratio, an important feature of image is the relationship between the widths and the height of the image. It can also be defined as the ratio between the horizontal and vertical dimensions of the image considered to be a major concern in broadcasting applications. Thus cropping video frames with specified rectangle changes the aspect ratio of the image disturbing the watermark .Visimark1_0 incorporated this major geometrical attack for testing the robustness of the video.

• 2.    Rotation:-

• The major geometrical attack is to change the dimensions of the video to a specified angle.VisiMark1_0 performs rotations in clockwise and anticlockwise directions. For cropping and rotation VisiMark1_0 uses the Matlab feature imrotate that implies nearest neighbor interpolation to fine-tune the enlarged size of the image after rotation since interpolation mechanism works by using known data to estimate values at unknown points. Nearest neighbor is the most fundamental and entail least processing time of all the interpolation algorithms. This is because it considers only one pixel that is neighboring to the interpolated point having the effect of making each pixel bigger.

• 3.    Affine attack:-

• Geometrical attacks category also includes the affine attack for evaluating the robustness of a watermarking algorithm. The affine transform preserves the colinearity and ratios of the distances. Co linearity could be termed as all points on a line initially, still lie on a line after the transformation. Ratios of the distance could be clarified by stating that midpoint of a line segment remains the midpoint after transformation.

• 4.    Scaling:-

• Shift in spatial or temporal direction in the video frames is scaling. VisiMark1_0 utilizes the nearest neighbor interpolation adapted by Matlab to scale the video frames.

• 5.    Blurring:-

• Blurring an image generally compose the image unfocused .In Visimark1_0, the frames are blurred by filtering them with a rotationally similar Gaussian filter with standard deviation. VisiMark_1 also provides the evaluated results with varying values of deviation.

• 6.    Sharpening:-

• Sharpening      effect      is      opposite      of

• 7.    Translation:-

• The position of the image is changed in the translation attack proving it to be an attack for robustness checking. Under translation, an image undergoes a translation (βx, βy) and the original position of element (x1, y1) in the original is shifted to a new position (x2, y2) in the consequent output image.

• 8.    Contrast stretching:-

• Contrast enhancement is a two step procedure, Specify lower pixel value and upper pixel values of the desired range, then to improve the contrast of the image, the above desired range is achieved by stretching the intensity values of the pixels in the image. Commonly, these limits will just be the minimum and maximum pixel values that the image type concerned allows. For example for 8-bit gray level images the lower and upper limits could be 0 and 255. VisiMark1_0 uses the limits [0.01  0.99] to saturate two percent of the image

• 9.    Up/down Sampling:-

• Commonly used technique in image processing of maintain the same 2d representation of the image even after increasing the spatial resolution is up sampling . Down sampling is vice versa. Down sampling can generally be measured as pre filtering the signal with a linear filter and then decimating it by a down sampling factor. VisiMark1_0 implies up/down sampling attack using nearest neighbor interpolation.

• 10.    Dithering:-

• VisiMark1_0 accomplishes the dithering effect by initially converting the frames to NTSC color space and then increasing the apparent color resolution using dither function in Matlab that implies Floyd Steinberg’s error diffusion dither algorithm.

• 11.    Flipping:-

• VisiMark1_0 achieves this common geometrical attack by flipping the frames column wise left to right.

• 2.    Frame Denoising :-

• Usually, denoising of images is a three step process

Fi=α Fi where α is the scaling factor

blurring.VisiMark1_0 attains the sharpening effect in Matlab using contrast enhancement filter that is created from the negative of the laplacian filter with parameter alpha.

VisiMark1_0 compiles affine transform on the translation matrix subsequently executing translation along x and y axis dependent on the translation factor.

subsequently adjusting the intensity of the image.

VisiMark1_0 Signal processing attacks:-1.    Frame Noising:-

An effective watermarking attack is the noise attack. We have successfully implemented the attack using Gaussian and Salt and pepper noise.

• a.    The image is transformed into some domain where the noise component is more easily identified.

• b.    The noise is then removed by applying a threshold operation.

• c.    A noise free image is constructed by performing the inverse transform .

• 3.    Filtering attacks:-

• VisiMark1_0 executes symmetric filtering attack, Gaussian filtering attack and averaging filter attack for color videos with wiener filtering, adaptive filtering and median filtering attacks for gray scale.

• 4.    Pixel removal attack:-

• Removing certain pixels from the video frames may bring about disturbance in the watermark. VisiMark1_0 achieves separate even and odd pixel removal attacks for the video. The number of pixels to be removed is the decision of VisiMark1_0 at the moment, which later could be modified.

• 5.    Compression attack:-

• Compression is a strong attack implemented in the traditional software’s. It is a belief that compressing an image disturbs the watermark appearance in the image. Following this folklore VisiMark1_0 performed the Jpeg compression of the Video as a compression attack.

• 3. 2 Implementation Particulars of Visimark1_0:-
  
  a)    Structure of VisiMark1_0:-

• Fig 1 shows the structure of VisiMark1_0. The tool provides individual folders for input, output and attacks report. The input watermarked video must be placed in the Input folder of VisiMark1_0. After the execution of VisiMark1_0, the attacked videos will be located in the output folder as separate files in the output folder. This adds up to the provision of accessing individual attacked videos. The attacks report will be generated as a text file in the Attacks Report folder. Also the evaluated parameters and their respective graphs will be available in the graphs folder of VisiMark1_0. A frame wise comparison with all evaluated parameters will be available in the comparison folder

VisiMark1_0 also considers the adaptive noise removal attack using wiener filter but with the limitation of converting to gray scale.

Fig. 1: Structure of Visimark1_0

• b)    Options of scheme:-

• In view of the fact that VisiMark1_0 provides two methods for the attacks mentioned in 3.1, an option is provided for selecting entire frames of the video or only the key frames.

Key frame selection algorithm implemented in visimark1_0

• 1.    For locating the scene change in the video

• i)    Select two frames of the video Vi andVj

• ii)    Feature extraction is done by the sobel method that finds edges using the sobel approximation to the derivative. It returns edges at those points where the gradient of I is maximum. Edge comparison is done for the feature extracted frames by computing average intensity of the pixels in the selected frames.

[intsimg1, r1, c1] = aiv(Bimg1)

[intsimg2, r2, c2] = aiv(Bing2)

Bimg1 and Bimg2 are the binary images of size of Wi and Wj, with 1's where the function finds edges in image and 0's elsewhere. aiv returns the average intensity.

• iii)    For computing average intensity

• a)    I = Read image File

• b)    The size of the image file is stored in image_Row & image_Col

• c)    The block Size is stored in block_row and block_Col

• d)    Compute Block size = Image_Row/Block_Row

iXgrw inycois(i,/) = ^  ^ ((G — 1) * block) +

1:    i » block^(j — 1) • block + hj * block)

• e)    Compute average intensity AIV iXgcoi r-,:inyr™ ^,

Л» = l/(imgrow * imycol) /       /, (5(i-j)

• iv)    Compute the edge difference

• 2.    The set of first frames in individual scenes are considered as the key frames for VisiMark1_0. Fig 2 illustrates the scene change detection algorithm.

  

  Fig. 2: Scene change detection algorithm

  
  

intsimg (i,j)= intsimg1(i,j)-intsimg2(i,j)

for i = 0 to r1 and j = 0 to c1

Compute absolute sum of intsimg1

result=abs(sum(intsimgt2(:)))

if (result>threshold) then a Scene Change is detected.

• c)    Graphical User Interface.

A graphical user interface is the scrupulous requirement of the vertical market. VisiMark1_0 provides a simple graphical user interface that helps to percept the software sound. A provision is made in the GUI for attacking all the frames of the video or only the selected key frames. Also the evaluated results are available on a click.

Fig. 3: GUI for VisiMark1_0

• d)    Support for color

The sturdy facet of VisiMark1_0 is the support for color. The watermarked videos supplied for evaluating could be colored or gray scale.

• e)    Specifications and System Requirements

Entire coding of VisiMark1_0 is done in Matlab. A Matlab GUI is used for simplicity of the tool. So the basic requirement is barely Matlab. VisiMark1_0 has been tested on a windows XP machine with Matlab version 7 with the image processing toolbox.

f) Evaluation Parameters

The comparison of the original and the attacked video are done based on following parameters

i) MSE: Mean square error between two images I1(m,n) and I2(m,n) is

щСт, n) — /;(m, n)J*

M « N

M and N are the number of rows and columns in the input images, respectively. Mean-squared error depends strongly on the image intensity scaling ii) PSNR:-PSNR scales the MSE according to the image range. It is measured in decibels (dB).

PSNR = 10 fo5w[—]

iii) MSAD[13]:-The value of this metric is the mean absolute difference of the color components in the correspondent points of image.

iv) DELTA [13]:-The value of this metric is the mean difference of the color components in the correspondent points of image.

IV. Experimental Results

The input to VisiMark1_0 is a watermarked video. The video can be watermarked with the various watermarking algorithms accessible on numerous sites.VisiMark1_0 prepares four folders to display the results.

A) Output folder: All the attacked videos are stored in the output folder. This adds up to the convenience of accessing individual attacked video files as per requirement. Some of the frames of the attacked videos are shown in fig 3a as shown

Rotation attacked frame

Gaussian filter attacked

Flipping attacked

Pixel Removal attack

Jpeg compression attack

Scaling attack

Fig. 3a: Attacked frames of various videos

Frame dropping attacked

Averaging filter attacked

Frame swapping attack

Changing frame rate of video attack

Sampling attack

Row removal attack

• B)    Graph folder: - All the evaluated parameters graphs are stored in the graphs folder. Though MSE, MSAD, PSNR, DELTA and SSIM index graphs are evaluated for all attacks of the video, some are illustrated in fig 4.

Fig. 4a:

Fig. 4b:

Fig. 4c:

Fig. 4d:

Fig. 4: PSNR, MSE, DELTA and MSAD graphs

C) Comparison folder: This folder displays the comparison of the watermarked and the attacked video frames along with parameter listing. Separate videos could also be given as input for their frames comparison for the parameters irrespective of the attacks. Some of the outputs are included in fig 5 as shown below.

Fig. 5: Frames in comparison folder

• C)    Attacks Report Folder: The attacks report is prepared and stored in a text file in the attacks report folder. Figure 6 shows the snapshot of the attacks report generated after the Visimark1_0 runs.

Finally a comparison table elaborating features of Visimark1_0 regarding attacks is elaborated in the Table 1 below.

Table 1: Comparison with available tools and benchmark

Attacks And Evaluated Parameters	SirMark [8] and [9] for images	Optimark [10] For png images	Checkmark [11] For images	Visimark1_0 For videos [Designed]
Frame Attacks:
Frame Averaging	NA	NA	NA	Yes
Frame dropping	NA	NA	NA	Yes
Frame swapping	NA	NA	NA	Yes
Changing the sequence of video	NA	NA	NA	Yes
Changing frame rate of the video	NA	NA	NA	Yes
Chroma_sampling attack	NA	NA	NA	Yes
Fade and dissolve attack	NA	NA	NA	Yes
Motion blurring attack	NA	NA	NA	Yes
Interframe filtering	NA	NA	NA	Yes
Geometric Attacks
Scale	Yes	Yes	Yes	Yes
Crop	Yes	Yes	Yes	Yes
Rotation	Yes	Yes	Yes	Yes
Dithering	-	-	Yes	Yes
Contrast Stretching	-	-		Yes
Rotation scale	Yes	Yes	Yes	Yes
Rotation Crop	Yes	Yes	-	Yes
Rotation/scaling/cropping	-	Yes	-	Yes
Aspect ratio	-	-	Yes	Yes
Shear	-	Yes	Yes	Yes
Linear	-	Yes	Yes	Yes
Affine	Yes	-	-	Yes
Blurring	-	-	-	Yes
Sharpening	-	Yes	-	Yes
Bending	-	-	Yes	-
Warping	-	-	Yes	-
Projective	-	-	Yes	Yes
Collage	-	-	Yes	-
Template	-	-	Yes	Yes
Row/column Removal	-	Yes	Yes	Yes
Up/down sampling	-	-	Yes	Yes
Signal Processing Attacks
Noise	Yes	Yes		Yes
Filtering	Yes	Yes	Yes	Yes
Compression Attack	Yes	Yes	yes
Evaluated parameters	PSNR	PSNR	PSNR	PSNR , MSAD MSE ,DELTA
GUI	No	Yes	Yes	Yes

Fig. 6: Snapshot of attacks report

• V. Conclusion

Visimark1_0 has been successfully designed as an assistant tool for evaluating the robustness of video watermarking algorithms by providing results of the watermarked video as desired by the researchers. The achieved results are fair enough to assist the researcher in the tedious process of evaluation. Currently the tool successfully presents the results of 9 video watermarking attacks related to video parameters like motion, frame rate etc. and twenty signal processing attacks and geometrical attacks for the individual frames of the video for avi file. An interactive strong and interactive GUI helps in the evaluation process. Further releases of VisiMark1-0 would provide video watermarking attacks like collision with a support for various video file formats like mpeg etc. Further plan also includes the evaluation of the extracted watermarks by parameters like the normalized correlation and the structural similarity index. Also a provision of default embedding and extraction algorithm for watermarking would be provided in near future.

Acknowledgements

We acknowledge our sincere thanks to authors of [11] for keeping the checkmark tool open for the research community. Also a sincere gratitude to the authors of [8 and 9] for providing Stirmark as a pioneer tool open for the research community as well. Certain guidelines were adapted from the files provided in the above tools for extension to video watermarking algorithm.